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PREFACE 


This  is  the  final  report  for  Task  Order  No.  6  of  Contract  No. 
F33615-88-C-0003 .  The  report  is  the  joint  effort  of  the  Air 
Force  Human  Resources  Laboratory  (AFHRL) ,  the  sponsor  of  the 
research,  ana  uhe  contractor,  Mei  Associates,  Inc. 

Section  1,  describing  the  history  of  the  AIDA  concept  within 
AFHRL  and  the  Laboratory's  long  range  plan  for  AIDA,  was  written 
by  AFHRL. 

Sections  2  and  3,  describing  the  purpose,  detailed  requirements, 
approach,  and  plan  for  Task  6,  were  prepared  by  Mei  Associates. 

Section  4,  containing  summaries  of  the  concept  papers,  synopses 
*  of  the  concept  design  and  review  meetings,  and  Section  5, 

defining  the  problems  to  be  addressed  in  follow-on  tasks,  were 
prepared  by  AFHRL. 

Section  6  presents  the  preliminary  system  specifications  that  are 
the  product  of  the  task  effort.  The  section  is  divided  in  two 
parts.  The  first  describes  the  functional  architecture  and  was 
written  by  AFHRL.  The  second  part  lists  a  sample  of  some  100 
system  functions  and  characteristics  defined  in  the  task.  It  was 
compiled  by  Mei  Associates. 

Finally,  Section  7  outlines  the  research  questions  which  were 
identified,  for  both  near  term  and  long  term  resolution  by  AFHRL. 
This  section  was  prepared  by  AFHRL. 

Mei  Associates  compiled  the  appendices  and  edited  the  complete 
manuscript . 

Dr.  Dan  Muraida  was  the  contract  monitor  for  AFHRL.  Dr.  Mike 
Spector  conceptualized  the  AIDA  project  under  the  direction  of 
Dr.  Scott  Newcomb,  Branch  Chief  for  the  Training  Technology 
Branch  of  the  Training  Systems  Division  of  AFHRL  (AFHRL/ I DC) . 

Dr.  Albert  E.  Hickey  was  project  manager  for  the  contractor,  Mei 
Associates . 
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SECTION  1 .  BACKGROUND 


The  purpose  of  this  section  is  to  provide  background  information. 
The  discussion  is  divided  into  four  parts:  1)  a  short  history  of 
the  AIDA  concept,  2)  AFHRL/IDC's  long  range  plan  for  AIDA,  3)  the 
long  range  schedule,  and  4)  the  benefits  of  the  long  range  AIDA 
program. 


1.1  HISTORY  OF  THE  AIDA  CONCEPT 

The  Advanced  Instructional  Design  Advisor  (AIDA)  was  first 
described  in  Preliminary  Design  Considerations  for  an  Advanced 
Instructional  Design  Advisor.  Dr.  Michael  Spector's  final  report 
for  his  1988  Summer  Faculty  Research  Program  grant  at  the  Air 
Force  Human  Resources  Laboratory  (AFHRL) .  That  research  was 
conducted  under  the  supervision  of  Dr.  Scott  Newcomb,  Branch 
Chief  for  the  Training  Technology  Branch  of  the  Training  Systems 
Division  (AFHRL/ I DC) . 

Newcomb  assigned  Spector  the  task  of  conceptualizing  a  next- 
generation  courseware  authoring  system  for  the  Air  Force. 

Spector  conducted  an  extensive  literature  search,  reviewed 
several  authoring  systems  used  by  the  DoD  including  the  Air 
Force's  ISS  and  SOCRATES  and  the  Navy's  AIM  and  CBESS  systems, 
and  had  many  engaging  discussions  with  Newcomb  and  other 
researchers  about  what  the  future  held  in  store  for  courseware 
authoring. 

The  central  problem  in  courseware  authoring  was  identified  as  the 
difficulty  and  expense  of  designing  effective  instructional 
materials  given  the  complexities  of  advanced  hardware  and 
software  technologies  and  the  variety  of  instructional  settings. 
Existing  systems  did  not  address  the  issue  of  effective 
instructional  design  at  the  course  level. 

Spector  proposed  a  computer-based  tool  to  assist  in  the 
instructional  design  process.  The  tool  would  reduce  course 
development  time  while  assisting  in  the  production  of 
consistently  effective  instructional  materials.  AIDA  would 
incorporate  prescriptive  advice  about  course  authoring  based  on 
established  theories  of  knowledge,  learning,  and  instruction. 

AIDA  would  contain  a  variety  of  tools.  Some  would  automate 
established  processes.  Others  would  assist  or  advise  authors 
about  designing  effective  instructional  materials.  The  tools 
might  be  used  as  stand-alone,  special  purpose  tools,  or  as 
ir.tegr-'ed  tools  using  a  shared  database  of  course  and  content 
information . 

Since  a  variety  of  tools  were  envisioned,  it  was  recommended  that 
a  modular  approach  be  adopted  and  that  a  standard  design 
philosophy  be  specified  in  order  to  provide  for  the  graceful 
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growth  of  the  system.  In  order  to  incorporate  the  best 
instructional  knowledge  available,  Spector  recommended  using  a 
team  of  experts  in  the  fields  of  epistemology,  cognitive 
psychology,  artificial  intelligence,  computer  systems,  and 
curriculum  and  instructional  design.  The  team  of  experts  would 
develop  specifications  for  a  standard  design  philosophy  and  a 
requirements  specification  for  AIDA.  A  prototype  AIDA  based  on  a 
minimal  functional  subset  would  then  be  built  and  evaluated. 

AFHRL/IDC  decided  to  continue  the  exploratory  development  of  AIDA 
under  Work  Unit  1121-10-43,  Computer-Based  Training  (CBT) 

Software  Development  and  Technical  Support.  The  AIDA  project  is 
primarily  a  response  to  the  Air  Training  Command  (ATC)  MPTN  89- 
14T ,  Research  and  Development  of  Computer-Based  Instruction 
(CBI) . 

In  a  follow-on  1989  Research  Initiation  Program  grant,  Spector 
submitted  a  report  entitled  Refinement  Considerations  for  an 
Advanced  Instructional  Design  Advisor.  In  that  document,  the 
instructional  design  process  was  divided  into  three  phases:  1) 
front  end  analysis  (FEA) ,  2)  design,  development,  and  delivery 
( DDD) ,  and  3)  rear-end  analysis  (REA) .  One  finding  was  that  the 
design  phase  of  DDD,  as  well  as  FEA  and  REA,  had  received 
inadequate  support  in  the  form  of  research  and  development. 

Spector  also  evaluated  the  potential  role  for  artificial 
intelligence  (AI)  in  the  instructional  design  process.  AI 
applications  were  divided  into  two  large  groups:  artificial 
neural  networks  and  expert  systems.  While  neural  networks  hold 
great  promise  in  the  general  area  of  pattern  recognition,  there 
does  not  appear  to  be  an  immediate  application  in  the  area  of 
instructional  design.  As  automated  learning  environments  become 
more  interactive  and  conversational,  perhaps  neural  networks  will 
have  a  significant  role  to  play  in  the  area  of  speech  processing. 

There  does  appear  to  be  a  significant  role  for  expert  systems  in 
instructional  design.  Diagnostic  expert  systems  have  already 
been  successfully  incorporated  into  intelligent  tutoring  systems. 
Expert  planning  systems  were  envisioned  to  aid  courseware 
designers  in  the  development  of  consistently  effective  course 
materials  for  a  variety  of  subject  matter  domains  and  knowledge 
types . 

Task  6  is  the  first  in  a  series  of  tasks  (see  section  1.2  below) 
to  refine,  elaborate,  and  evaluate  the  AIDA  concept.  This  report 
reflects  progress  on  AIDA  through  May  1990.  The  AFHRL/IDC  AIDA 
Project  Manager  is  Dr.  Dan  Muraida.  The  contractor  for  Task  6  is 
Mei  Associates,  Inc.  of  Lexington,  Massachusetts.  Mei 
Associates'  Principal  Investigator  is  Dr.  Albert  Hickey.  Spector 
has  continued  working  with  AFHRL  on  the  AIDA  project  under  a 
University  Resident  Research  Program  grant. 
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THE  LONG  RANGE  PLAN  FOR  AIDA 


This  section  represents  AFHRL/IDC's  current  overarching  plan  for 
the  AIDA  project. 

The  purpose  of  the  Advanced  Instructional  Design  Advisor  (AIDA) 
program  is  to  provide  reliable,  effective,  and  efficient 
instructional  design  guidelines  for  Air  Force  course  designers. 
The  approach  to  tnis  goal  contains  four  thrusts.  Each  is 
described  below. 


1.2.1  Status  and  Structure  of  Instructional  Design  Knowledge 

The  first  thrust  is  an  effort  to  assess  what  is  currently  known 
about  instruction  and  instructional  design.  This  assessment 
focuses  on  the  major  validated  findings  of  instructional  design 
research.  The  purpose  is  to  derive  critical  instructional  design 
information  and  initial  estimates  of  the  necessary  detail  for 
instructional  design  guidelines.  In  addition,  this  effort  will 
identify  areas  of  contradictory  research  findings  and  produce 
recommendations  for  an  instructional  design  research  and 
development  agenda.  This  work  was  conducted  under  a  task  order 
contract  (Task  6) . 


1.2.2  Instructional  Design  Guidelines:  Functional  Requirements 

The  second  thrust  attempts  to  identify  optimal  methods  of 
organizing  and  presenting  validated  instructional  design 
information  in  the  form  of  guidelines.  Part  of  this  effort 
builds  on  prior  work  which  will  have  identified  relationships 
among  different  areas  of  instructional  design  knowledge. 

The  focus  is  on  functional  requirements  for  an  organized 
collection  of  guidelines  which  accurately  represent  the 
complexities  of  instructional  design  without  exposing  the  user  to 
those  complexities.  The  functional  requirements  will  be  based  on 
a  model  of  instructional  design  which  embraces  the  processes  of 
design,  development  and  delivery.  A  subsequent  task  will 
elaborate  the  functional  requirements  and  develop  a  set  of  design 
specifications  for  an  automated  system  of  guidelines.  This  work 
was  conducted  under  two  task  order  contracts  (Tasks  6  and  13) . 

The  computer  architecture  and  human  factors  issues  involved  in 
producing  automated  guidelines  will  be  studied  in  a  Small 
Business  Innovative  Research  (SBIR)  effort. 
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1.2.3  Empirical  Studies  of  Instructional  Guidelines  Delivery  and 
Supporting  Instructional  Strategies 

In  the  third  thrust,  empirical  research  will  be  planned  in 
parallel  with  the  development  of  elaborated  functional 
requirements  and  specifications.  The  elaborated  functional 
requirements  and  specifications  will  encompass  a  complete 
automated  guideline  system,  although  they  will  be  used  to  build 
an  experimental  testbed  system  which  implements  only  certain 
basic  functions  of  an  automated  guideline  system. 

All  empirical  research  done  prior  to  the  development  of  the 
experimental  testbed  system  will  be  conducted  under  the  auspices 
of  an  in-house  work  unit.  All  development  work  devoted  to 
building  the  testbed  will  be  conducted  under  a  Broad  Agency 
Announcement  (BAA)  contracting  vehicle.  Empirical  research  using 
the  testbed  system  to  conduct  studies  on  guideline 
characteristics  or  instructional  strategies  will  be  conducted 
under  the  auspices  of  the  BAA. 


a .  Prior  to  Completion  of  the  Experimental  Testbed: 

Guidelines  research  will  focus  on  identifying  critical  variables 
in  the  automated  instructional  design  process.  Existing 
prototypes  which  automate  a  limited  number  of  basic  instructional 
design  processes  will  be  used  for  this  purpose.  The  data  and  the 
conclusions  of  this  stage  of  the  guidelines  research  will  provide 
useful  information  for  design  of  the  experimental  testbed  system 
and  the  subsequent  research  which  it  will  support. 

Research  in  the  effectiveness  of  instructional  strategies  with 
computer-based  interactive  technologies  will  also  employ  existing 
instructional  authoring  tools.  The  first  set  of  instructional 
strategy  studies  will  address  the  lack  of  instructional 
principles  for  the  use  of  audio  reinforcement  in  a  CBT  setting. 

A  second  set  of  studies  will  examine  Merrill's  transaction  theory 
and  the  usefulness  of  transaction  shells  in  CBT.  The  results  of 
the  work  accomplished  prior  to  the  completion  of  the  experimental 
testbed  system  will  add  to  the  knowledge  base  of  instructional 
design  principles  required  for  a  system  of  instructional 
guidelines. 


b .  After  Completion  of  the  Experimental  Testbed: 

At  this  point  most  of  the  guidelines  research  will  be  conducted 
using  the  testbed's  capability  to  present  different  types  of 
instructional  design  guidance  and  different  methods  of 
interacting  with  the  user.  Likewise,  most  of  the  instructional 
strategy  research  will  use  the  capability  of  the  testbed  system 
to  implement  basic  instructional  functions  (e.q.,  establishing 
linkages  between  instructional  objectives  and  plausible 
instructional  strategies) . 
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The  research  and  development  conducted  during  this  phase  of  the 
AIDA  project  will  provide  new  information  about  the  optimal 
nature  and  delivery  characteristics  of  instructional  design 
guidance  for  the  non-expert  instructional  designer.  Secondly, 
this  research  phase  will  begin  to  produce  conclusions  about  the 
extent  to  which  instructional  strategies  (new  or  old)  can  be 
extrapolated  to  a  computer-based  interactive  setting. 


1.2.4  Empirical  Test  of  an  Instructional  Design  Guidance 
Mer.hodoloqy 

The  fourth  thrust  of  the  AIDA  research  program  will  consist  of 
the  design,  development,  and  testing  of  the  complete  experimental 
model.  This  will  include  tests  of  all  functions  of  the  automated 
guideline  system,  continued  experimentation  on  optimal  guideline 
characteristics,  and  continued  research  on  instructional 
strategies  in  the  automated  delivery  of  :nstruction.  This  work 
will  be  conducted  under  a  fully  specified  contract. 


1 • 3  THE  LONG  RANGE  SCHEDULE 


1.3.1  Milestones 

Complete  work  in  first  thrust  area  with  the  following  products; 

Review  of  current  instructional  design  theory; 

A  cognitive  model  of  the  instructional  design  process; 
Functional  requirements  for  instructional  design  guidelines; 
A  research  and  development  agenda  for  instructional  design; 
Recommendations  for  instructional  design  research. 

. 3rd  QTR9  0 


Business  Strategy:  Task  Order  Contract 
Contract  award  for  functional  and  design  specifications  for  a 


subsystem  of  instructional  design  guidelines  ...3rd  QTR90 
Business  Strategy:  Task  Order  Contract 
Completion  of  instructional  design  guidelines  . 1st  QTR91 


Contract  award  for  (1)  design,  development  and  testing  an 
experimental  model  of  an  automated  guideline 
system  and  (2)  initial  instructional  strategies  and 
guidelines  research  . 1st  QTR91 

Business  Strategy:  Broad  Agency  Announcement 

Completion  of  experimental  model  tests  and  associated 

instructional  strategies  and  guidelines  studies ...  1st  QTR94 
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Contract  award  for  (1)  Construction  and  test  of  the 

complete  experimental  model  of  automated  guideline 
system  and  (2)  continuation  of  instructional  strategies 
and  guidelines  research  . 1st  QTR94 

Completion  of  experimental  model  tests  and  instructional 

guidelines  and  strategies  research.  . 1st  QTR98 


1.3.2  List  of  Relevant  Work  Units 


Thrust: 

1st : 
2nd 
3rd 
4  th 


1121-10-43  , 
1121-10-43  , 
1121-10-66 , 
1121-10-67  , 


Task  6 

Task  6  and  13;  1121-10-70  (SBIR) 

In-House  Work  Unit;  1121-10-71  Direct  BAA 
Fully  Specified  Contract. 


1  -  4  BENEFITS  OF  THE  LONG  RANGE  AIDA  PROGRAM 


1.4.1  Pavof  f 

The  expected  contributions  of  AIDA  to  Air  Force  training  are: 
o  reduced  training  costs 

o  increased  utility  of  CBT  technologies 

o  increased  instructor  productivity 

o  reduced  student  time  under  instruction 

o  increased  student  comprehension  and  learning  transfer 
o  establishment  of  instruction  standards 

o  improved  quality  assurance. 


1-4.2  Opportunity 

Subject  Matter  Experts  (SMEs)  with  little  or  no  experience  in 
instructional  design  or  ISD  will  be  able  to  use  a  theoretically 
oriented  and  empirically  validated  collection  of  instructional 
design  tools  to  determine  optimal  instructional  designs.  As  a 
result,  novice  instructional  designers  will  be  able  to  produce 
efficient  and  effective  instruction  in  a  variety  of  delivery 
modes . 
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SECTION  2.  INTRODUCTION  TO  TASK  6 


2 . I  PURPOSE  OF  TASK  6 

The  objective  of  Task  6,  called  AIDA  Phase  I,  was  to  design, 
develop,  and  document  the  concept  and  functional  specifications 
for  AIDA  as  the  basis  for  subsequent  work  units  (see  Sec.  1.2.1). 


2 . 2  BACKGROUND 

Basic  behavioral  science  issues  had  to  be  resolved  before 
designing  AIDA.  For  example:  (1)  Which  theories  of  knowledge 
might  be  useful  in  the  instructional  design  process?  (2)  Which 
learning  theories  and  taxonomies  might  be  useful?  (3)  Which 
instructional  design  theories  and  models  might  be  useful?  i4) 
what  is  the  potential  role  of  artificial  intelligence  in 
instructional  design?  There  werf  other  issues  to  be  resolved. 

For  example:  (1)  Which  strategies  should  be  followed  in  order  to 

design  and  develop  advanced  authoring  aids?  (2)  What  kinds  of 
advanced  authoring  aids  already  exist  and  how  effective  are  they? 
(3)  How  mignt  ISD  be  incorporated  into  AIDA?  These  were  the 
questions  to  be  answered  in  Task  6. 
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SECTION  3.  METHODOLOGY  FOR  TASK  6 


To  accomplish  Task  6,  Mei  Associates  retained  as  consultants 
seven  of  the  most  creative  people  in  the  fields  of  psychology  ar.d 
instructional  design,  both  theory  and  practice,  and  brought  them 
together  in  an  interactive  procedu~“  which  resulted  in  the 
integration  of  the  best  concepts  for  AIDA. 


3 . 1  THE  CONSULTANTS 

Seven  consultants  were  chosen  from  among  academic  researchers 
with  a  reputation  and  demonstrated  research  in  fields  related  to 
instruction  desb'  a.  They  are: 


1 . 

Or. 

Robert 

M 

.  Gagne 

Florida  State  University 

2  . 

Dr. 

Henry  M 

Hal  f  f 

Halff  Resources,  Inc. 

3  . 

Dr. 

M .  Da  v 

Merrill 

Utah  State  University 

4  . 

Dr. 

Martha 

C 

Poison 

University  of  Colorado 

5  . 

Dr. 

Robert 

D 

.  Tenryson 

University  of  Minnesota 

6  . 

Dr. 

Harold 

F 

.  O'Neil 

University  of  Southern  California 

7  . 

Dr. 

Charles 

Re igeluth 

Indiana  University 

The  first  five  consultants  on  the  list  were  designated  "design 
consultants."  The  last  two  were  appointed  "review  consultants" 
to  review  and  critique  the  concept  documents  prepared  by  the 
design  consultants. 

Abbreviated  vicae  cf  the  seven  consultants  are  presented  in 
Appendix  E. 


3 • 2  THE  MILITARY  ADVISORS 

At  the  same  time,  the  Air  Force  Human  Resources  Laboratory 
( AFHRL)  invited  specialists  from  the  three  services  to  join  a 
board  of  Military  Advisors.  The  military  advisors  are: 


1. 

Dr.  Dee  Andrews 

AFHRL/OTT 

2  . 

LCOL  Jerry  Barucky 

USAFRS/RSCD 

3  . 

LCOL  Mike  Bush 

USAFA/DFF 

4  . 

LCOL  Larry  Clemons 

ATC/XPCRI 

5  . 

CAPT  James  Coward 

HQ  ATC/XPCRI 

6. 

Mr.  Brian  Dallman 

3400  TCHTW/TTOZLCOL 

7  . 

LCOL  Mike  Dickinson 

HQ  HSD/YA 

8  . 

Dr.  John  A.  Ellis 

NPRDC 

9  . 

Mr.  John  LaBarbera 

HQ  USAF/DPPT 

10  . 

Dr.  Mary  Marlino 

USAFA/DFT 

11 . 

Dan  Meigs  (Retired) 

3  3  02d  TCHTS/CC 

12  . 

MAJ  Robert  Mongillo 

ATC/XPCRR 

13  . 

Rich  Ranker 

AWC/DFP 

14  . 

MAJ  Karen  Reid 

ATC/TTIP 

15  . 

Dr.  Robert  Seidel 

Army  Research  Inst. 
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3 . 3  PROJECT  MANAGEMENT 


The  Project  Manager  for  the  contractor,  Mei  Associates,  was  Dr. 
Albert  E.  Hickey.  The  Project  Monitors  for  AFHRL  were  Drs. 
Daniel  Muraida  and  Michael  Spector. 


3 . 4  THE  PROCEDURE 

As  described  in  detail  in  Sections  1  and  2,  the  objective  of  Task 
6  was  to  design,  develop,  and  document  the  concept  and  functional 
specifications  for  AIDA. 

The  task  was  divided  into  two  consecutive  design  cycles,  each 
about  four  months  in  duration.  Each  design  cycle  was  organized 
around  three  two-day  meetings  at  AFHRL:  a  Planning  Meeting,  a 
Concept  Design  Meeting,  and  a  Concept  Review  Meeting.  The 
schedule  is  shown  in  Table  3.1. 


Cyc 

le  1 : 

1 . 

Kick-off  Meeting,  Part  1 

Jul 

6 

Part  2 

Jul 

18-19 

Part  3 

Aug 

8-9 

2  . 

First  Concept  Design  Meeting 

Oct 

18-19 

3  . 

First  Concept  Review  Meeting 

Dec 

5 

Cycle  2 : 


4  . 

Planning  Meeting 

Dec 

6 

5  . 

Second  Concept  Design  Meeting 

Feb 

20-21 

6. 

Final  Concept  Review  Meeting 

Apr 

24-25 

TABLE  3 . 1  PHASE  I  SCHEDULE 
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Figure  3.1 


AFHRL  and  Mei  Associates  collaborated  in  formulating  the  agenda 
for  the  six  meetings.  The  meeting  agenda  are  reproduced  in 
Appendix  A. 

In  each  of  the  two  design  cycles,  the  five  design  consultants 
performed  tasks  assigned  during  the  first  meeting  in  the  cycle. 

In  cycle  1,  the  tasks  were  tailored  to  the  individual 
consultants.  In  cycle  2,  all  five  design  consultants  received 
the  same  tasking  letter  and  were  asked  to  choose  the  remaining 
issues  they  wished  to  address.  The  design  consultants  submitted 
their  task  results,  in  the  form  of  concept  papers,  to  Mei 
Associates  for  distribution  to  the  participants  prior  to  each 
Concept  Design  Meeting.  The  papers  were  presented  and  discussed 
at  the  meeting.  Some  papers  were  subsequently  revised. 

At  each  of  the  two  Concept  Design  Meetings,  the  five  design 
consultants  drafted  the  tasking  letter  for  the  two  review 
consultants,  Drs.  O'Neil  and  Reigeluth.  In  cycle  2,  two  of  the 
military  advisors,  Mr.  Dallman  and  Dr.  Marlino,  also  accepted 
review  assignments. 

The  review  consultants  were  asked  to  allocate  five  days  to  their 
review  of  the  concept  paper  and  the  ensuing  discussion.  Their 
reviews  were  disseminated  in  draft  prior  to  each  Concept  Review 
Meeting,  where  they  were  presented  and  discussed. 

Copies  of  the  tasking  letters  to  the  design  and  review 
consultants  are  reproduced  in  Appendix  B. 

The  proceedings  of  all  meetings  were  recorded  and  transcribed  by 
the  contractor.  AFHRL  prepared  and  distributed  a  Progress  Report 
following  each  meeting.  A  list  of  the  documents  produced  during 
the  task  is  given  in  Appendix  C. 
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SECTION  4.  TASK  RESULTS  AND  CONCLUSIONS 


The  purpose  of  this  section  of  the  report  is  to  summarize  the 
results  of  the  current  task,  Task  6.  The  discussion  is  divided 
into  five  parts:  1)  Analyses  of  the  cycle  1  papers,  2)  Analyses 
of  the  cycle  2  papers,  3)  Reviews  of  the  meetings,  4)  Areas  of 
consensus  and  disagreement,  and  5)  Conclusions  about  key 
parameters  and  issues  related  to  AIDA. 


4 . 1  ANALYSES  OF  CYCLE  1  PAPERS 

This  section  contains  summaries  of  the  papers  submitted  in 
cycle  1.  Specific  task  assignments  for  these  papers  can  be 
found  in  Appendix  B. 


4-1-1  Gagne's  Cycle  1  Paper:  "Principles  of  Instructional 
Theory" 


Summary 

Gagne's  purpose  was  to  cite  the  basic  principles  established  in 
instructional  science  and  indicate  their  importance  in 
determining  the  nature  of  instruction.  Gagne  outlined  12 
principles  of  instructional  theory  and  indicated  the  relative 
importance  and  difficulty  of  learning  each.  Discussion  indicated 
general  agreement  with  two  notable  exceptions:  1)  Those  indicated 
as  easy  to  master  were  not  so  easy  to  master,  and  2)  the 
presentation  (not  the  order)  of  generality,  instances/non¬ 
instances,  and  practice  was  the  critical  element  in  the  principle 
concerning  concepts  and  rules.  Gagne  revised  his  paper  to 
reflect  his  agreement  on  these  two  issues. 


Major  Assumptions 

1.  Single  objective  lesson  components  can  be  isolated. 

2.  Applying  instructional  principles  will  optimize  learning. 

3.  Mastery  of  many  instructional  principles  is  relatively  easy. 

4.  There  is  no  general  principle  covering  expository  or 
discovery  learning. 

5.  Learning  objectives  determine  instructional  strategies. 

6.  The  nature  of  the  cognitive  task  determines  the  learning 
objective . 

7.  Learning  is  internal;  instruction  is  external. 
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Major  Propositions 

With  regard  to  student  attention: 

1.  Arouse  student  interest  before  initiating  any  other 
transactions . 

With  regard  to  task  clarity: 

2 .  Communicate  clearly  what  the  student  must  learn  to  do  — 
demonstrate  its  usefulness. 

3 .  Communicate  the  relation  between  what  is  to  be  learned  and 
its  usefulness. 

4.  Make  the  stimulus  aspect  of  the  task  readily  perceptible. 
With  regard  to  stimulating  cognitive  processing: 

5.  Stimulate  recall  of  previously  learned  relevant  information. 

6.  Guide  learning  through  elaborations. 

7.  Verify  initial  learning  by  performance. 

8.  Provide  varied  practice  with  corrective  feedback. 

9.  Arrange  occasions  that  require  retrieval. 

With  regard  to  learning  objectives: 

10.  Five  types  of  learning  objectives  have  been  identified. 

They  are  coupled  with  the  following  instructional 
strategies : 

(i)  Verbal  -  Relate  to  organized  prior  knowledge;  use 
spaced  review. 

(ii)  Concepts  -  Provide  definitions,  examples,  non¬ 
examples,  and  practice  in  identifying  new  examples. 

(iii)  Procedural  -  Verify  mastery  of  parts  of  a  task  before 
attempting  the  whole  task. 

(iv)  Motor  -  Provide  practice  with  reinforcement. 

(v)  Attitudes  -  Provide  demonstration  with  a  human 
role  model. 
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4.1.2  Halff's  Cycle  1  Paper:  "Prospects  for  Automating 
Instructional  Design" 


Summary 

Halff's  purpose  was  to  provide  a  concept  of  how  AIDA  would 
support  the  development  of  two  broad  instructional  paradigms,  and 
to  identify  the  principles  of  learning  and  instruction  that  apply 
to  the  design  of  each  of  the  AIDAs  and  to  its  instructional 
products.  Halff  argued  that  cognitive  structures  have  a  role  to 
play  in  instructional  design.  He  maintained  that  learning 
mechanisms  and  instructional  design  are  interdependent;  a 
shortcoming  of  many  current  instructional  design  researchers  and 
implementers  is  to  ignore  learning  mechanisms. 

Halff  described  two  general  approaches  to  the  automation  of 
instructional  design:  advisory  and  generative.  The  advisory 
approach  attempts  to  automate  the  formulation  of  instructional 
designs,  making  implementation  primarily  the  responsibility  of  a 
human  developer.  The  generative  approach  attempts  to  use  the 
computer  to  generate  instruction  from  known  instructional 
paradigms.  He  then  presented  two  examples  of  generative 
approaches  and  argued  that  generative  approaches  to  automating 
instructional  design  show  some  promise. 

Halff  asserted  that  successful  automation  of  instructional  design 
requires  a  prudent  division  of  labor  between  the  human  and  the 
computer.  He  saw  the  large  scale  production  of  instructional 
materials  which  conform  to  complex  specifications  as  an 
appropriate  role  for  the  computer  in  instructional  design. 

Another  role  would  be  the  use  of  the  computer  to  track  the 
development  of  the  design,  offering  suggestions  on  modifications 
and  additions.  He  argued  that  the  interpretation  of 
instructional  design  principles  and  the  decisions  on  how  to 
fashion  them  into  a  viable  design  should  lie  within  the 
jurisdiction  of  the  human  developer. 


Major  Assumptions 

With  regard  to  instructional  theory  and  learning  theory: 

1.  A  theory  of  instruction  is  needed  to  support  both  automation 
and  justification  of  the  instructional  design  functions. 

2.  Learning  should  be  understood  in  mechanistic  terms. 

3.  A  complete  theory  of  learning  will  describe  stimuli, 
responses,  performance  mechanisms,  and  the  laws  of  learning. 

4.  The  individual  organism  is  the  unit  of  analysis  in  any 
theory  of  learning. 
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With  regard  to  knowledge  representation,  transmission,  and 

context : 

5.  Cognitive  functions  receive  contextual  support  in  non- 
instructional  situations. 

6 .  As  knowledoe  evolves  so  do  the  mechanisms  for  its 
transmission  (i.e.,  instruction). 

7.  Instruction  is  an  evolutionary  process. 

8.  The  non-instructional  context  is  significant. 

With  regard  to  instructional  design: 

9.  In  any  given  context  the  implementation  of  successful 
instructional  design  recommendations  is  a  function  of  the 
designer's  interpretation  and  intuitions. 

10.  Instructional  design  principles  are  not  separable  from 
content. 

with  regard  to  ISD: 

11.  ISD  implies  that  instructional  objectives  can  be  determined 
without  reference  to  instructional  methods  and  that  methods 
are  not  subject  specific.  (Halff  does  not  support  this 
separation. ) 


Major  Propositions 

Concerning  instruction  and  instructional  design: 

1.  Effective  instructional  design  is  a  cooperative  enterprise 
between  instructional  designers  and  learners  based  on  mutual 
observance  of  cultural  conventions  for  effective  learning. 

2.  Instructional  designers  are  typically  not  committed  to 
learning  mechanisms  and  theory,  but  they  are  sensitive  to 
the  situated  nature  of  learning. 

3.  Human  instructional  designers  should  be  involved  in  some  way 
in  the  development  of  instruction. 

4 .  Top-down  structure  analysis  for  instructional  design  works 
when  applied  appropriately. 

With  regard  to  prospects  for  automation: 

5.  Computers  can  be  usefully  employed  to  generate  alternative, 
device-specific  strategies,  sequence  problems,  project 
costs,  and  evaluate  them. 
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6. 


Automation  should  assist  in  the  generation  of  instruction 
from  known  designs,  and  should  support  existing 
instructional  paradigms  for  broad  subject  areas,  rather  than 
attempting  to  implement  a  universal  theory  of  instruction. 


4.1.3  Kintsch's  Cycle  1  Paper:  "Principles  of  instructionax 
Theory  from  Research  on  Human  Cognition" 


Summary 

Kintsch's  paper,  which  she  prepared  under  a  subcontract  to 
Poison,  presented  10  learning  principles  drawn  primarily  from 
cognitive  science  research  (especially  Glaser  and  Bassok,  1989) 
with  emphasis  on  their  implications  for  instruction.  These 
principles  were  well  received  by  the  other  consultants, 
reflecting  general  consensus  in  this  area.  The  principles  are  as 
follows : 

1.  Comprehension  in  all  domains  is  an  active  process  of  meaning 
construction  by  which  the  learner  creates  an  interpretation 
of  the  incoming  information.  Instruction  should  be  centered 
on  the  learner's  own  knowledge  construction  activities. 

2.  Learning  occurs  by  extending  existing  knowledge.  In 
designing  instruction,  one  needs  to  be  aware  where  students 
are  in  acquiring  new  skills,  and  what  kind  of  knowledge 
manipulation  is  important  in  a  given  context. 

3.  The  organization  of  knowledge  in  memory  determines  its 
accessibility  and  the  depth  of  understanding  that  is 
achieved.  Instruction  should  find  multiple  ways  to  tie  new 
information  to  the  learner's  prior  knowledge. 

4.  Learning  is  enhanced  when  knowledge  is  learned  in  a  specific 
context  with  a  specific  and  stated  cognitive  goal.  New 
knowledge  should  be  taught  in  a  meaningful  context. 

5.  Comprehension,  writing,  problem  solving  and  learning  are 
strategic  rather  than  rule-governed  processes.  Instruction 
should  be  designed  to  make  the  relevant  cognitive  strategies 
explicit. 

6.  Metacognition  is  a  significant  component  of  knowledge 
acquisition  and  use.  Cognitive  apprenticeship  is  a 
promising  instructional  approach. 

7.  Knowledge  acquisition  is  supported  by  learning  in  a  social 
setting.  Cooperative  and  collaborative  learning 
environments  should  be  fostered,  even  in  CBT  settings. 
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3.  The  ability  to  engage  in  higher-level  thinking  depends  on 
automaticity  at  lower  levels.  Instruction  should  provide 
opportunities  for  extensive  practice. 

9.  Learning  is  enhanced  through  immediate  qualitative  feedback. 
Instruction  should  provide  feedback  geared  to  the  learner's 
present  level  of  knowledge  or  skill. 

10.  The  content  of  instruction  should  be  designed  to  fit  the 
changing  needs  of  students  at  different  stages  of  skill 
acquisition.  Tasks  should  increase  in  complexity; 
sequencing  should  reflect  increasing  diversity;  and  tasks 
requiring  global  skills  should  be  presented  first  to 
encourage  the  construction  of  a  conceptual  model. 

There  is  no  unified  learning  theory  to  support  these  10 
principles,  but  the  author  believes  that  it  is  important  to 
design  instruction  within  a  theoretical  account  of  human 
information  processing.  The  result  of  applying  these  principles 
would  be  less  emphasis  on  didactics  and  more  emphasis  on  problem 
solving  in  a  collaborative  setting. 


Major  Assumptions 

1.  Attention  to  mental  structures  proposed  by  cognitive 
scientists  when  designing  instruction  can  improve  learning. 

2.  There  are  three  distinct  types  of  knowledge:  declarative 
knowledge  of  a  subject  domain,  procedural  knowledge  for 
operating  with  declarative  knowledge,  and  knowledge  about 
the  conditions  of  knowledge  use  (executive  control/cognitive 
strategies) . 

3.  Learning  is  an  active  process  of  meaning  construction  or 
interpretation. 

4.  Learning  is  strategic  —  not  rule-governed. 

5.  The  organization  of  knowledge  determines  its  accessibility. 

6.  Qualitative  differences  in  strategies  separate  experts  from 
novices . 

7.  Internal  memory  structures/representations  are  transformed 
in  knowledge  transformation. 


Major  Propositions 

1.  See  the  10  principles  presented  in  the  summary;  some 
corollaries  follow. 
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2.  Design  instruction  to  encourage  the  learner's  independent 
meaning  construction  activities. 

3.  Introduce  connectedness  with  new  knowledge  to  promote 
retention  and  transfer. 

4.  Introduce  new  knowledge  in  a  meaningful  context. 

5.  Maximize  learner  involvement. 

6.  Put  the  learner  in  control  of  the  learning  process. 

7.  Teach  strategies  m  service  of  meaningful  goals. 

4.1.4  Merrill's  First  Cycle  1  Paper:  "Project  AIDA:  A  Concept 
Paper" 


Sumrcrv 

Merrill's  purpose  was  to  define  the  concept  of  AIDA,  to  indicate 
the  functions  it  should  perform,  including  knowledge  acquisition 
and  strategy  analysis,  and  to  identify  the  principles  of 
knowledge  theory,  learning  theory,  and  instructional  theory  that 
underlie  these  functions.  This  paper  is  divided  into  three 
parts:  1)  background  information  on  Merrill's  ID  EXPERT  vl.O  and 
v2 . 0 ,  2)  a  conceptual  review  of  the  components  of  ID  EXPERT  v3.o, 
and  3)  a  discussion  of  the  theoretical  foundations  of  ID  EXPERT 
v3 . 0 . 

ID  EXPERT  vl.O  was  funded  by  the  Army  Research  Institute  (ARI ) 
and  implemented  using  the  expert  system  shell  S.l  on  a  VAX 
computer.  ID  EXPERT  v2.0,  partially  funded  by  ARI,  was 
implemented  using  NEXPERT  and  HYPERCARD  on  Macintosh  SE 
computers.  ID  EXPERT  v3 . 0  is  being  developed  based  on  the 
premise  that  previous  systems  and  instructional  design  theories 
(ID-1)  lack  prescriptions  appropriate  to  advanced  interactive 
technologies. 

ID  EXPERT  v3 . 0  consists  of  4  main  subsystems:  1)  a  Knowledge 
Acquisition  and  Analysis  System,  2)  a  Strategy  Analysis  System, 

3)  a  set  of  transaction  subsystems,  and  4)  an  Intelligent 
Advisor-based  Instructional  Delivery  System.  Many  of  these 
subsystems  ar’  constructed  of  mini-experts  which  provide  highly 
specialized  expertise.  The  middle  part  of  the  paper  describes 
this  modular  conception  in  detail. 

The  last  part  of  the  paper  is  devoted  to  the  theoretical 
foundations  for  ID  EXPERT  v3.0.  A  second  generation 
instructional  design  theory  (ID-2)  is  proposed,  based  on  nine 
limitations  of  ID-1.  The  limitations  include  such  things  as  the 
emphasis  on  small  components,  failure  to  recognize  integrated 
wholes,  limited  prescriptions  for  course  organization,  and  the 


18 


general  insensitivity  to  new  interactive  technologies.  ID-2 
should  be  capable  of  analyzing,  representing,  and  guiding 
instruction  to  teach  integrated  sets  of  knowledge  and  skills;  it 
should  be  capable  of  producing  prescriptions  for  the  selection  of 
instructional  strategies  and  the  selection  and  sequencing  of 
instructional  transactions;  it  should  be  an  open  system,  able  to 
incorporate  new  knowledge;  and  it  should  integrate  the  phases  of 
instructional  development. 


Major  Assumptions 

1.  Learning  results  in  the  organizing  of  memory  into  structures 
called  mental  models. 

2.  There  are  different  learning  outcomes  and  different 
conditions  required  to  promote  these  outcomes. 

3.  The  details  of  cognitive  structures  are  not  well  understood. 


Major  Propositions 

1.  ID-1  had  a  behavioral  orientation;  a  cognitive  orientation 
is  needed. 

2.  Organization  during  learning  aids  in  later  retrieval. 

3 .  Elaborations  generated  at  the  time  of  learning  can 
facilitate  retrieval. 

4.  Different  learning  outcomes  require  different  types  of 
mental  models. 

5.  The  construction  of  a  mental  model  by  a  learner  is 
facilitated  by  instruction  that  explicitly  organizes  and 
elaborates  the  knowledge  during  the  instruction. 

6.  Different  learning  outcomes  require  different  organizations 
and  elaborations. 

7.  The  means  of  knowledge  representation  should  be  determined 
by  the  use  to  which  the  knowledge  will  be  put. 

8.  The  identification  of  instructional  goals  is  critical  to 
instructional  design. 

9.  Instructional  strategy  specifies  a  pedagogy  for  selecting, 
sequencing,  customizing,  and  integrating  instructional 
units.  There  are  levels  of  instructional  strategy.  Micro¬ 
strategy  is  embedded  into  a  transactional  unit  and  controls 
the  presentation  of  that  transaction.  The  next  level 
strategy  determines  how  transactions  are  assembled  into 
frame  sets.  A  higher  level  determines  how  frame  sets  are 
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integrated  into  elaborated  frames  corresponding  to  a 
particular  instructional  goal.  The  highest  level  integrates 
the  goals  for  a  course. 

10.  There  are  three  fundamental  frame  types:  entity,  activity, 
and  process. 

11.  There  are  three  types  of  elaborations:  component, 
abstraction,  and  association. 


4.1.5  Merrill's  Second  Cycle  1  Paper:  "Project  AIDA:  A  Theory 
Paper" 


Summary 

Merrill's  second  paper  was  generously  provided  to  facilitate 
understanding  of  the  principles  and  theory  underlying  the  first 
paper.  The  paper  began  with  a  definition  of  theory  and  a 
distinction  between  descriptive  and  prescriptive  theories. 
Theories  involve  objects  and  relationships.  Descriptive  theories 
are  concerned  with  identifying  and  describing  the  relevant 
objects.  Prescriptive  theories  are  more  concerned  with  the 
relationships  among  objects. 

The  next  section  of  the  paper  reviewed  ID-1  and  ID-2.  ID-1  is 
concerned  with  the  wrong  objects;  ID-1  objects  are  too  small  and 
fragmented.  ID-2  should  focus  on  more  robust  objects  such  as 
mental  models  and  integrated  interactions.  A  major  task 
confronting  ID-2  is  to  completely  describe  these  more  complex 
instructional  objects. 

Instructional  design  theory  must  resolve  two  major  questions:  (1) 
What  to  ceach  (content)?  and  (2)  How  to  teach  (pedagogy)?  The 
remainder  of  the  paper  is  devoted  to  assumptions  and  major 
propositions  relevant  to  these  two  concerns. 


Major  Assumptions 

1.  Instructional  theory  should  be  more  concerned  with 
descriptive  theories. 

2.  Memory  consists  of  both  declarative  and  procedural 
knowledge . 

3.  Memory  is  organized  into  integrated  sets  of  declarative  and 
procedural  knowledge  called  mental  models. 

4.  Different  learning  outcomes  require  different  conditions. 
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Major  Propositions 

1.  Organization  during  learning  aids  in  later  retrieval. 

2 .  Elaborations  generated  at  learning  time  can  facilitate 
retrieval . 

3  .  Instruction  should  focus  on  helping  learners  acquire  mental 
models . 

4.  Acquiring  mental  models  is  facilitated  by  instruction  that 
explicitly  organizes  and  elaborates  knowledge. 

5.  Significant  learning  outcomes  consist  of  complex  human 
activities  (enterprises)  requiring  integrated  sets  of 
knowledge  and  skill. 

6.  Different  enterprises  require  different  mental  models,  the 
cognitive  representation  of  the  knowledge  and  skill  required 
for  an  enterprise. 

7.  Different  learning  activities  are  required  for  promoting  the 
acquisition  of  different  mental  models. 

8.  Instructional  interactions  should  be  organized  around  all 
the  activities  needed  to  promote  acquiring  a  particular 
mental  model. 

9.  Use  integrated  interactions  focusing  on  all  of  the  knowleuga 
comprising  an  elaborated  frame  network  to  facilitate 
acquisition  of  a  mental  model. 

10.  Different  types  of  knowledge  frames  require  different  types 
cf  transactions. 

11.  The  following  six  content  rules  apply: 

(i)  There  are  limits  to  short-term  memory  (7  +/-  2). 

(ii)  Learning  the  attributes  of  a  class  is  essential  to 
generalization  and  transfer. 

(iii)  There  is  no  enterprise  without  an  entity. 

(iv)  Teaching  a  tool  requires  an  application. 

(v)  New  mental  models  are  built  from  previously  acquired 
mental  models. 

(vi)  A  process  underlies  every  human  activity  and  is  that 
which  provides  an  understanding  of  that  activity;  one 
can  execute  the  activity  without  knowing  the  process. 
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12.  The  following  12  pedagogy  rules  apply: 

(i)  Transactions  should  be  consistent  with  the  role  and 
function  type  of  knowledge  involved;  student 
expectations  should  be  ta.’.en  into  account;  different 
roles  require  instruction  that  serves  different 
functions. 

(ii)  Learners  follow  the  path  of  least  effort. 

(iii)  Maximize  the  amount  of  learner  control  that  can  be 
used  to  a  learner's  advantage. 

(iv)  Transactions  should  promote  active  interactions. 

(v)  Learners  learn  what  they  do. 

(vi)  Practice  without  performance  feedback  is  not 
productive . 

(vii)  There  are  three  primary  presentation  forms: 
generality,  instance,  and  practice;  transactions 
should  include  all  three. 

(viii)  Early  learning  should  involve  extensive  attention 
focusing. 

(ix)  Learning  improves  when  information  is  represented  in 
multiple  ways. 

(x)  Learning  improves  when  instances  and  non-instances 
are  carefully  matched. 

(xi)  Learning  improves  when  divergent  instances  are 
presented. 

(xii)  Learning  improves  when  simple  knowledge  and  skills 
are  elaborated  to  form  more  complex  knowledge  and 
skills . 


4 . 1 . 6  Muraida  and  Spector:  "Lowry  Needs  Assessment  Report" 


Summary 

Muraida  and  Spector  conducted  an  in-depth,  preliminary  needs 
assessment  at  the  Lowry  AFB  Technical  Training  Wing  (TTC)  2-6  Oct 
1989.  Brian  Dallman  (3400  TCHTW/TTOZ)  acted  as  facilitator  and 
arranged  interviews  with  instructional  designers,  developers, 
training  managers,  and  instructors  involved  in  various  CBT 
developments.  Capt.  Coward  (ATC/XPCRI)  participated  in  the 
interviews  and  contributed  many  useful  insights. 
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Major  Assumption 


1.  Air  Force  practitioners  have  many  useful  ideas  and  lessons 
learned  to  contribute  to  the  design  of  an  AIDA. 


Major  Propositions 

1.  There  is  a  general  lack  of  knowledge  about  educational 
principles  and  implications  for  instructional  design  at  all 
levels . 

2.  There  is  a  need  for  a  centralized  source  of  instructional 
design  expertise. 

3.  There  is  an  immediate  demand  and  need  for  tools  to  automate 
ISD  documentation  and  develop  course  materials.  Such  tools 
should  allow  users  to  focus  on  developing  and  revising 
materials  and  make  ISD  documentation  a  transparent  support 
system. 

4.  There  is  very  little  understanding  of  alternative  (i.e. 
non-instructor-based)  instructional  models. 

5.  Consistently  effective  decision-making  in  the  instructional 
design  process  is  needed.  Instructional  design/decision 
aids  which  offer  informed  alternatives  and  which  can  produce 
justifications  would  be  useful. 

6.  Decision  aids  should  be  aimed  at  both  informed  and 
uninformed  audiences  and  should  provide  justifications  and 
alternatives . 


4.1.7  Poison's  Cycle  1  Paper:  "Cognitive  Theory  as  a  Basis  for 
Instruction" 


Summary 

Poison's  purpose  was  to  state  the  major  distinguishing 
characteristics  of  the  contents  and  processes  of  cognition, 
including  the  kinds  of  knowledge  and  the  different  forms  each  may 
take  (i.e.,  summarize  the  results  of  cognitive  science  relevant 
to  AIDA) .  Cognitive  science  tends  to  model  human  cognition  as  an 
information  processing  system  operating  on  three  types  of 
knowledge:  declarative,  procedural,  and  contextual. 

The  key  components  of  human  cognition  are  working  memory  and  long 
term  memory.  Learning  is  a  matter  of  encoding  and  organizing 
information  so  that  it  can  be  retrieved  and  used  when 
appropriate.  Representation  systems  are  critical  in 
understanding  human  knowledge.  Knowledge  is  viewed  as  follows: 

1)  associative  in  nature,  2)  organized  into  functional  units,  3) 
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permeated  with  default  beliefs  and  potential  inconsistencies,  and 
4)  hierarchical  in  structure. 

The  paper  then  reviewed  two  well  developed  theories  of  cognition: 
1)  Kintsch's  construction-integration  model  of  discourse 
comprehension,  and  2)  Anderson's  ACT*  theory  of  skill 
acquisition.  Kintsch's  theory  is  concerned  with  how  declarative 
knowledge  structures  are  acquired.  Anderson's  theory  is  an 
attempt  to  model  the  acquisition  of  procedural  knowledge.  Both 
argue  for  computer-based  training  systems  to  support  their  views, 
although  Kintsch's  CBT  is  non-intelligent  while  Anderson  supports 
the  use  of  intelligent  tutoring  systems. 

The  conclusion  was  that  a  unified  theory  of  cognition  does  not 
exist,  although  Kintsch  and  Anderson  together  present  a  small 
step  in  that  direction.  Lack  of  a  unified  theory  should  not 
impede  progress,  however.  As  Glaser  and  Bassok  have  indicated, 
progress  in  an  area  is  often  made  on  the  basis  of  instrumentation 
that  is  designed  to  facilitate  experimentation.  With  regard  to 
AIDA,  a  significant  tool  would  be  the  design  of  instructional 
interventions  that  operationalize  proposed  theories  in  the  form 
of  environments,  techniques,  materials,  and  equipment. 


Major  Assumptions 

1.  All  assumptions  for  Eileen  Kintsch  apply. 

2.  Working  memory  is  limited  and  requires  active  processing  to 
maintain  information;  long  term  memory  is  unlimited  and  does 
not  require  active  processing  to  maintain  information. 

3.  In  order  for  a  cognitive  theory  to  be  useful  to 
instructional  design  it  will  have  to  provide  guidance  on  how 
to  design  instruction  to  foster  encoding  that  facilitates 
appropriate  retrieval  and  manipulation. 

4.  People  act  to  maintain  their  goals  through  rational  action. 

5.  Information  processing  is  an  active  process  on  complex 
knowledge  structures;  intelligent  action  results  from  such 
processes . 

6.  Learning  is  the  use  of  strategic  processing  to  encode, 
organize,  retrieve,  and  transform  information. 

7 .  A  theory  of  knowledge  must  be  based  on  knowledge 
representation;  issues  of  representation  play  a  key  role  in 
all  theories  of  cognition  and  memory. 

8.  A  proposition  represents  a  single  idea;  propositions 
represent  units  of  meaningful  knowledge. 
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The  goal  of  instruction  should  be  to  promote  coherent, 
interrelated,  well-structured,  easily  accessible  knowledge 
units . 


9 . 


Major  Propositions 

1.  All  of  Kintsch's  propositions  apply. 

2.  The  first  task  in  instructional  design  is  to  analyze  the 
knowledge  structures  of  the  expert  so  that  the  instruction 
can  be  aimed  at  promoting  the  correct  structures. 

3 .  Types  of  knowledge  do  not  map  directly  onto  types  of 
representation . 

4.  Learning  is  the  successive  transformation  of  naive  mental 
models  into  a  series  of  increasingly  complex  mental  models 
adequate  for  solving  larger  sets  of  problems. 

5.  We  know  more  about  the  differences  between  novices  and 
experts;  we  know  less  about  the  mechanisms  of  transition. 

6.  Useful  representations  for  declarative  knowledge  include 
propositions,  semantic  nets,  scripts,  schemata,  frames,  and 
plans. 

7.  The  most  common  form  of  representing  procedural  knowledge  is 
a  production  system. 

8.  Control  processes  and  structures  exist  which  guide  the 
processing  of  information;  this  includes  metacognitive 
knowledge. 

9.  It  is  too  early  in  the  history  of  cognitive  science  to  have 
a  complete  theory  of  instruction. 


4.1.8  Tennyson's  Short  Cycle  1  Paper:  Cognitive  Model  of  Learning 


Summary 

Tennyson's  short  paper  presents  the  cognitive  model  that  is  used 
in  the  longer  paper  to  update  ISD.  Tennyson's  cognitive  model  of 
learning  stresses  that  knowledge  acquisition  comes  from  internal 
as  well  as  from  external  sources;  behavioral  models  allowed  only 
for  external  sources.  Tennyson  defines  a  knowledge  base  as  an 
associative  network  of  concepts.  Knowledge  bases  vary  with 
regard  to  amount,  organization,  and  accessibility.  It  is  the 
organization  and  accessibility  of  knowledge  that  distinguishes 
excerts  from  novices,  not  the  amount  of  knowledge. 


He  identifies  three  types  of  knowledge:  declarative  ("knowing 
that"),  procedural  ("knowing  how"),  and  contextual  ("knowing  when 
and  why").  It  is  contextual  knowledge  which  determines  the 
organization  and  accessibility  of  the  other  two  types  of 
knowledge . 

Tennyson  goes  on  to  identify  three  cognitive  abilities: 
differentiation  (selection) ,  integration  (restructuring) ,  and 
creation.  These  cognitive  abilities  are  used  by  the  human 
retrieval  system  in  the  service  of  three  thinking  strategies: 
recall,  problem  solving,  and  creativity.  This  constitutes 
Tennyson's  operational  definition  of  cognitive  complexity. 


Major  Assumptions 

1.  Instructional  theories  are  theories  which  prescribe 
manipulations  of  instructional  variables  and  conditions 
hypothesized  to  improve  learning. 

2.  A  learning  need  or  prob] em  is  part  of  a  larger  environment; 
contexL  and  connectedness  are  crucial  to  understanding. 


Major  Proposition 

1.  A  major  consideration  for  instructional  design  is  how 

specific  knowledge  will  be  stored,  retrieved,  and  used  in 
problem  solving. 


4.1.9  Tennyson's  Long  Cycle  1  Paper:  Cognitive  Science  Update 
of  Instructional  Systems  Design  Models 


Summary 

The  long  paper  is  a  detailed  and  thorough  update  of  ISD  taking 
into  account  the  contributions  from  computer  science,  management 
information  science,  psychology,  and  instructional  technology. 

ISD  refers  to  a  system  of  procedures  to  guide  the  development  of 
learning  environments.  Instruction  involves  both  the  planning  of 
an  environment  in  which  learning  can  occur  and  the  delivery  of 
information  in  that  environment.  Early  ISD  models  were  based  on 
behavioral  learning  theory  (e.g.,  small  steps  of  content 
presentation,  overt  learner  responding,  and  immediate 
reinforcement  of  correct  responses)  and  designed  according  to 
principles  of  general  systems  theory  (e.g.,  behavioral 
objectives,  pretest,  organization,  post-test,  revision) . 

Updated  ISD  (UISD)  is  broken  into  five  phases:  analysis,  design, 
development,  implementation,  and  maintenance.  Evaluation  is  made 
a  key  item  in  each  phase.  The  analysis  phase  draws  heavily  on 
learning  theory  and  emphasizes  the  connectedness  of  human 
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knowledge.  Older  ISD  models  fail  to  adequately  integrate 
knowledge.  Knowledge  engineering  techniques  are  recommended  to 
extract  expert  knowledge  and  structure  an  appropriate  knowledge 
base  for  a  particular  domain.  The  analysis  phase  ends  with  an 
instructional  design  decision.  Tennyson  recommends  documenting 
this  decision  in  the  form  of  a  proposal  that  states  the  rationale 
for  the  instrv.cticuai  .uecisiciiS  and  specifics  the  means  for 
accomplishing  the  stated  goals. 

Learning  objectives  are  identified  at  the  beginning  of  the  design 
phase.  Tennyson's  proposed  categories  are:  1)  verbal 
information  (facts,  concepts,  rules,  and  principles  — 
declarative  knowledge) ,  2)  intellectual  skills  (using  concepts, 
rules,  and  principles  —  procedural  knowledge) ,  3)  contextual 
skills  (organization  and  accessibility  of  a  knowledge  base) ,  4) 
cognitive  strategies  (elaboration  of  strategies  to  provide 
students  with  more  detailed  domain-specific  contextual  knowledge, 
and  development  of  integration  and  differentiation) ,  and  5) 
creative  processes  (the  ability  to  solve  problems  derived  from 
the  outside  and  the  ability  to  create  new  problems) . 

A  critical  step  in  the  design  phase  is  the  analysis  of 
information  organization.  Tennyson  proposes  three  forms  of 
elaborated  analysis:  1)  attribute  characteristics  (for 
declarative  knowledge) ,  2)  semantic  structure  (connections  to 
prerequisite  knowledge) ,  and  3)  schematic  structure  (connections 
within  and  among  the  schemata  of  a  given  domain  of  knowledge. 
Instructional  strategies  should  be  sensitive  to  both  the 
information  to  be  learned  and  the  lesson  objective.  He 
identifies  five  categories  of  instructional  prescriptions:  1) 
expository  strategies,  2)  practice  strategies,  3)  problem- 
oriented  strategies,  4)  complex-dynamic  strategies,  and  5)  self- 
directed  experiences.  The  other  phases  of  UISD  reflect  similar 
modifications . 


Manor  Assumptions 

With  regard  to  instructional  design: 

1.  Instructional  design  is  a  system  of  procedures  to  guide 
development  of  effective  learning  environments. 

2.  The  behavioral  paradigm  for  organizing  knowledge  is  rot 
adequate  for  sequencing  instruction. 

With  regard  to  ISD: 

3.  Procedures  defined  in  ISD  models  are,  for  the  most  part, 
supported  by  instructional  theories. 

4 .  Research  and  theory  in  cognitive  science  and  instructional 
technology  can  provide  the  means  to  update  the  learning 
theory  foundations  of  the  ISD  model. 
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5. 


The  major  phases  of  the  ISD  and  USID  process  are  analysis, 
design,  development,  implementation,  and  maintenance. 


Major  Propositions 

1 .  Learning  problems  should  be  related  to  the  total  learnina 
context  in  terms  of  links  to  hierarchically  and  laterally 
related  domains. 

2.  It  should  be  possible  to  trace  each  instructional  activity 
back  through  the  curricular  system  to  specific  needs  and 
goals  of  the  curriculum. 

3.  Evaluation  should  be  an  integral  part  of  all  ISD  variables. 

4.  Curricular  design  variables  should  be  integrated  into  the 
development  of  instruction. 

5.  There  is  a  need  to  investigate  variables  and  conditions  for 
testing  and  evaluating  domain-specific  higher  cognitive 
processes . 

6.  AIDA  should  include  the  capability  to  assist  designers  and 
developers  in  identifying  appropriate  instructional  and 
human  resources. 


4-1.10  O'Neil's  Cycle  1  Critique 


Summary 

The  general  thrust  of  O'Neil's  critique  was  that  AIDA  was  an 
ambitious  and  important  project  with  serious  funding  problems. 

As  a  result,  not  all  of  the  ideas  suggested  for  AIDA  would  be 
able  to  be  realistically  incorporated.  For  example,  given 
current  budgetary  constraints  AIDA  would  not  be  able  to  provide 
support  for  all  of  ISD;  how  much  of  the  earlier  and  later  phases 
of  ISD  to  support  would  need  to  be  resolved.  In  addition,  AI  is 
expensive;  therefore,  a  serious  issue  is  how  to  take  advantage  of 
AI  in  AIDA  and  contain  costs  at  the  same  time. 

O'Neil  suggested  that  the  AFHRL/IDC  AIDA  Project  Team  should 
prioritize  capabilities  desired  in  AIDA.  In  support  of  this 
suggestion,  he  offered  an  instrument  to  use  in  rating  22  possible 
AIDA  system  features.  This  instrument  was  used  at  the  April  AIDA 
review  meeting.  Results  are  summarized  in  Appendix  D. 

O'Neil  recommended  the  technique  of  working  backwards  from  cost 
constraints  to  the  features  to  be  included.  He  suggested  that  it 
was  unreasonable  to  expect  to  be  able  to  reduce  time  and  cost  of 
producing  courseware  and  also  improve  the  quality  of  the 
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courseware.  Again,  someone  would  have  to  decide  which  was  most 
important . 

The  working  assumptions  behind  O'Neil's  remarks  were  the 
following:  1)  CBI  and  CBT  were  or  could  be  effective,  2) 

Authoring  is  the  bottleneck  in  CBT,  3)  Current  authoring  systems 
do  not  focus  on  instructional  design,  and  4)  Some  ISD  model  was 
desirable . 

With  regard  to  requirements,  O'Neil  said  that  an  important  aspect 
of  AIDA  would  be  the  ability  to  provide  justifications  for 
decisions  made  in  designing  and  developing  instruction.  In 
addition  to  the  needs  assessment  performed  at  Lowry  AFB,  some 
study  of  existing  authoring  aids  should  be  conducted.  AFHRL/IDC 
does  have  another  project  called  Computer-based  Selection  and 
Implementation  Strategies  (CSIS)  which  will  include  an  analysis 
of  various  authoring  languages  used  by  the  Air  Force.  In 
considering  the  design  of  AIDA,  some  consideration  should  be 
given  to  the  Air  Force  of  tomorrow. 

With  regard  to  implementation,  O'Neil  strongly  emphasized  the 
need  to  look  at  existing  systems  and  develop  a  set  of  lessons 
learned.  The  particular  systems  that  O'Neil  recommended  for 
review  were  the  following:  1)  AIS  II/ISS,  2)  PLATO/TUTOR,  3) 
COURSE  OF  ACTION,  3)  NPRDC 1 S  AIM,  4)  AFHRL'S  IKAT,  5)  IDIOM,  6) 
Air  University's  SOCRATES,  7)  IMTS,  and  8)  numerous  commercial 
systems.  Merrill  quickly  amended  the  list  to  include  his  own  ID 
Expert,  Elron  Corporation's  ACE,  the  Alberta  Research  Council's 
CMI ,  Xerox  PARC's  IDE,  System  G,  Ford  Aerospace's  HUMORS,  and 
several  others  to  be  described  in  the  March  90  issue  of 
Educational  Technology.  O'Neil  indicated  that  one  thing  such  a 
review  would  suggest  was  the  important  trade-off  between 
ease-of-use  and  power. 

Another  task  that  O'Neil  suggested  worth  pursuing  was  the 
development  of  baseline  data  concerning  costs  and  time  of 
producing  lesson  materials  with  the  various  systems.  His  own 
experience  indicates  roughly  400  hours  and  $10K  per  hour  of 
individualized  instruction  with  wide  variations  due  to  a  variety 
of  authoring  systems,  media  technologies,  and  constraints. 

O'Neil  identified  three  approaches  to  CBT:  1)  artistic,  2) 
analytic,  and  3)  empirical.  The  artistic  approach  is  used  by 
media  developers  who  emphasize  action  and  dramatic  content.  The 
analytic  approach  uses  accepted  instructional  theorems  to  guide 
the  development  of  specific  materials.  The  empirical  approach  is 
the  means  by  which  the  other  two  approaches  can  validate  claims 
that  their  methods  are  effective. 

O'Neil  proposed  the  following  collection  of  tools  which  could  be 
incorporated  into  AIDA:  1)  automation  of  existing  paperwork,  2) 
task  analysis,  3)  cognitive  task  analysis,  4)  simulation  support, 
5}  t-omplates  of  varying  levels  of  power  and  flexibility  which  are 
sensitive  to  types  of  knowledge  and  control  strategies,  and  6) 
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computer-generated  assignments.  O'Neil  identified  the  tool  most 
in  demand  as  the  automation  of  existing  paperwork.  O'Neil  also 
described  a  typical  macro-instructional  sequence  with  much 
emphasis  on  test  and  measurement.  He  indicated  his  belief  that 
test  and  measurement  should  be  made  an  integral  component  of  AIDA 
and  supported  the  point  with  a  description  of  four  generations  of 
computerized  measurement.  He  also  provided  several  sample 
templates  to  illustrate  how  templates  could  be  used  to  guide 
lesson  development. 

O'Neil  then  presented  three  sets  of  issues  for  AIDA:  management, 
technical,  and  authoring.  Management  issues  included  the 
decision  to  support  individual  or  team  authoring,  novice  or 
experienced  instructional  designers,  provide  tools  or  an  entire 
system,  provide  programming  aids  or  design  aids,  support  a 
limited  or  full  range  of  media  presentation  possibilities,  and 
pursue  a  fixed  price  or  cost  plus  contracting  mechanism. 

Technical  issues  included  categorizing  types  of  learning  relevant 
to  instructional  design,  identifying  underlying  theories  of 
learning  (behavioral  and  cognitive)  and  their  implications, 
specifying  the  influence  of  cognitive  science  and  its 
implications  for  instructional  design,  and  performing  formative 
evaluation  with  various  media. 

The  major  issue  with  regard  to  author  interaction  was  to  decide 
which  types  of  interactions  to  support,  given  a  choice  of 
templates,  customized  parameters  with  templates,  programmable 
templates,  on-line  help,  on-line  advice,  intelligent  job  aiding, 
and  articulate  expertise. 

He  also  presented  a  list  of  four  different  ways  that  AI  could  be 
used  in  AIDA:  1)  Intelligent  Computer-Assisted  Instruction,  2) 
Intelligent  Job  Aid,  3)  Intelligent  Interface,  and  4)  Intelligent 
Help.  O'Neil  argued  that  the  most  promising  choice  was  the 
third,  as  this  approach  allowed  an  intelligent  front-end  to 
templates,  which  would  most  likely  be  included  in  some  form  in 
AIDA. 

O'Neil's  main  point  with  regard  to  these  issues  was  that  any 
choice  had  significant  consequences  with  regard  to  the  power  and 
cost  of  AIDA. 


4.1.11  Reioeluth's  Cycle  1  Critique 


Summary 

Reigeluth's  critique  focused  on  this  question:  What  is  likely  to 
contribute  to  a  workable  AIDA?  His  paper  is  divided  into 
suggestions  in  four  areas:  1)  approaches  to  designing  AIDA,  2) 
conceptions  of  how  AIDA  should  operate,  3)  the  nature  of  required 
knowledge  bases,  and  4)  miscellaneous  issues. 
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With  regard  to  approaches,  Reigeluth  made  a  distinction  between 
prescriptive  theories  of  instruction  (science  of  the  artificial) 
and  descriptive  theories  of  learning  (science  of  the  natural) . 
Inductive  methodologies  derive  prescriptive  theories  of 
instruction  from  practice.  Deductive  methodologies  derive 
prescriptive  theories  of  instruction  from  theories  of  learning. 

Reigeluth  recommended  following  an  inductive  approach,  not 
attempting  to  deduce  instructional  prescriptions  from  learning 
theories.  He  indicated  that  rapid  prototyping  was  an  effective 
way  to  implement  an  inductive  approach. 

Reigeluth  also  suggested  that  the  initial  experimental  prototype 
should  be  aimed  at  a  simple  task  not  involving  generating 
instruction.  Reigeluth  suggested  that  the  AIDA  design  approach 
should  be  based  on  existing  recognized  and  relevant  principles  of 
instruction,  without  trying  to  develop  ID-2  first.  He  suggested 
that  the  initial  development  effort  would  guide  the  development 
of  ID-2.  Following  the  logic  of  this  approach,  Reigeluth 
suggested  relying  heavily  on  field  testing  of  the  experimental 
subsystem  to  evaluate  both  quality  of  instruction  produced  and 
the  ease  of  use  of  the  system. 

With  regard  to  how  AIDA  should  work,  Reigeluth  argued  that  it 
should  be  an  assistant,  not  just  an  advisor,  in  the  sense  that 
AIDA  should  do  much  of  the  work  of  the  developer,  generating 
instruction  from  known  designs  (templates) .  AIDA  would  be  used 
by  both  an  SME  and  a  designer.  The  instructional  designer  would 
review  the  instruction  generated  by  the  computer  for  optimality. 
Reigeluth  saw  a  continuing  role  for  an  instructional  designer  to 
handle  content-specific,  cultural,  and  contextual  considerations 
that  cannot  be  automated. 

With  regard  to  the  nature  of  the  knowledge  base,  Reigeluth  argued 
that  learner  variables  should  not  be  overemphasized  as  they  only 
influence  what  should  be  taught  rather  than  how  material  should 
be  taught.  Reigeluth  argued  that  front-end  analysis  should  not 
be  tied  to  a  single  theory  of  learning. 

Reigeluth  argued  that  the  sequence  of  activities  would  be  to 
decide  which  instructional  strategies  to  incorporate  in  the 
initial  experimental  subsystem.  These  instructional  strategies 
would  dictate  the  structure  of  particular  instructional 
development  models  to  be  incorporated  into  the  system.  He  made  a 
distinction  between  types  of  knowledge  and  levels  of  cognitive 
ability  for  domain  dependent  content  that  he  found  missing  in 
Tennyson's  paper.  The  same  type  of  knowledge  could  be  learned  at 
a  memorization  level,  at  an  application  level,  or  at  an 
understanding  level.  The  level  of  cognitive  ability  desired 
influences  the  methods  of  instruction  likely  to  optimize 
attainment  of  the  learning  objective. 
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For  memorization  tasks,  Reigeluth  argued  that  the  most  useful 
distinction  is  between  associations,  lists,  ordered  lists,  and 
lists  of  associations.  For  application  tasks,  Reigeluth  argued 
that  the  important  distinctions  are  between  concepts,  principles, 
and  procedures.  For  understanding,  the  critical  distinction  is 
between  conceptual  and  causal  understanding.  Conceptual 
understanding  requires  distinguishing  types  of  relationships, 
each  of  which  indicates  a  different  instructional  strategy.  Why 
these  are  the  important  distinctions,  who  makes  them,  and  how 
they  are  made  was  not  indicated.  Whether  or  not  other  meaningful 
levels  of  cognitive  ability  are  useful  was  not  examined. 

Reigeluth  argued  that  domain  independent  content  did  not  have 
associated  with  it  different  levels  of  cognitive  levels. 
Consequently,  it  was  acceptable  to  speak  of  a  single  category  of 
strategies  —  cognitive  strategies. 

Reigeluth  believes  that  it  is  important  to  distinguish  contextual 
and  causal  knowledge  (both  domain  dependent)  from  cognitive 
strategies  (domain  independent) .  He  indicated  that  we  know  more 
about  instructional  strategies  for  causal  knowledge,  some  about 
teaching  cognitive  strategies,  and  little  about  teaching 
contextual  knowledge. 

The  last  section  of  the  paper  contained  miscellaneous 
observations  and  suggestions.  He  claimed  that  the  major  issue  in 
instructional  theory  concerns  optimality,  not  validity.  The 
distinction  between  natural  (descriptive)  and  artificial 
(prescriptive)  may  be  in  dispute,  but  there  is  general  agreement 
that  AIDA  should  be  aimed  at  devising  instructional  strategies 
which  do  optimize  learning  in  terms  of  retention,  transfer,  time 
to  learn,  and  depth  of  understanding. 
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4 . 2 


This  section  contains  summaries  of  the  papers  submitted  in 
cycle  2. 

4.2.1  Friedman's  Cycle  2  Paper:  "Designing  Graphics  for 

Courseware:  The  Role  of  Graphics  in  Knowledge  and  Skill 

Acquisition" 


Summary 

The  paper  by  Alinda  Friedman,  prepared  under  a  subcontract  with 
Poison,  was  generally  highly  regarded  and  well  received. 

Friedman  in  effect  suggested  that  building  a  mini-expert  for 
incorporating  graphics  effectively  into  courseware  involved 
sufficiently  complex  research  and  development  issues  to  justify  a 
separate  project.  Her  review  and  analysis  of  the  relevant 
literatures  is  definitely  worthwhile.  One  suggested  taxonomy  of 
graphics  uses  the  realistic/ abstract  distinction  as  the  primary 
dimension,  with  photographs  at  one  end  of  the  spectrum  and  tables 
and  charts  at  the  other. 

Another  way  to  categorize  graphics  is  according  to  information 
content  (e.g.,  shape,  color,  spatial  location,  action  sequence, 
etc.).  An  information  parser  could  then  make  recommendations 
regarding  the  types  of  representations  best  suited  to  each  need. 

A  domain  parser  could  conceivably  provide  recommendations  about 
general  conveyance  needs  to  be  used  by  the  information  parser. 


4.2.2  Gagne's  Cycle  2  Paper:  AIDA  Concept  of  Operation" 


Summary 

Gagne  identified  several  key  assumptions  that  he  had  made:  1) 
the  instructional  designers  using  AIDA  would  be  academically 
capable  SMEs  with  some  minimal  instructional  design  experience  or 
exposure,  2)  users  would  be  capable  of  performing  a  job  task 
analysis  or  job  tasks  would  already  be  categorized  into  learnable 
capabilities,  3)  it  is  desirable  to  have  instructional  designers 
follow  standard  procedures  that  are  adaptable  to  a  variety  of 
trainees,  environments,  and  tasks,  and  4)  it  is  desirable  to 
develop  an  advisory  system  for  this  purpose. 

Gagne's  AIDA  requires  the  designer  to  make  intelligent  decisions 
and  use  common  sense.  He  sees  military  designers  and  developers 
using  AIDA  to  create  instruction,  guided  by  its  advice.  AIDA 
will  not  create  the  instruction  on  its  own,  as  suggested  in 
Merrill's  "moonshot"  system. 

Gagne's  AIDA  used  Merrill's  tentative  architecture  (Information, 
Content,  Executive,  Strategies,  and  Delivery) ,  and  proceeded  to 
specify  the  relevant  considerations  and  characteristics  in  each 
component's  area.  The  information  component  would  indicate 
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whether  target  trainees  were  readers  or  non-readers,  whether  the 
environment  was  instructor-led  or  computer-based,  and  what 
stimulus  modes  of  the  task  to  be  trained  were  involved. 
Instruction  was  divided  into  large  periods  or  stages:  setup 
period,  initial  presentation,  and  practice.  Possible  strategies 
were  suggested  for  each  stage  as  follows: 

Stage  A  (set-up) :  1.  grabber 

2 .  scenario 

3.  reminder 

4.  recall 

Stage  B  (presentation) :  1.  statement 

2.  example 

3 .  label 

4.  mnemonic 

5.  discourse 

6.  analogy 

7.  elaboration 

8.  human  model 

9 .  question 

Stage  C  (practice):  1.  practice 

2 .  assessment 

3.  telling 

4.  transfer 

Gagne  maintained  that  content  should  be  categorized  by  five  types 
of  capabilities:  intellectual  skills,  verbal  information, 
cognitive  strategies,  motor  skills,  and  attitudes.  Gagne  made  it 
a  point  to  distinguish  what  is  learned  (performance  capabilities) 
from  the  content  of  a  presentation.  Gagne  believes  that  content 
is  overemphasized  at  the  expense  of  learned  capabilities. 

In  Gagne's  AIDA,  it  is  the  type  of  capability  more  than  any  other 
factor  that  determines  the  selection  of  specific  instructional 
strategies.  Gagne  lists  four  steps  involved  in  instructional 
strategy  selection:  1)  choose  the  media,  2)  establish 
integrative  control,  3)  classify  the  single  capabilities  to  be 
learned,  and  4)  select  and  order  appropriate  instructional 
strategies  for  each  capability. 

Gagne  claimed  that  the  notion  of  an  enterprise  scenario  was 
useful  and  he  encouraged  its  elaboration.  However,  he  admitted 
that  it  is  not  known  exactly  how  to  get  from  an  enterprise 
scenario  to  individual  capabilities,  but  that  it  was  not  a  random 
process;  rather,  he  suggested  following  some  unspecified  natural 
order. 

Everyone  seemed  willing  to  accept  most  of  what  Gagne  presented. 

In  fact,  one  way  to  construe  Gagne's  contribution  is  as  the 
rational  middle  ground  —  whatever  path  gets  explored  or  whatever 
system  evolves,  Gagne's  viewpoints  should  be  taken  into 
consideration. 


4.2.3  Halff's  Cycle  2  Taper: 
and  Development" 


At  tomat 1 ng  Instructional  Design 


Sumna rv 

Halff  began  with  a  review  of  his  interpretation  of  the  AIDA 
concept.  Put  simply,  AIDA  is  a  description  in  information 
processing  terminology  of  the  design,  development,  and  delivery 
of  instruction.  He  distinguished  ♦'wo  kinds  of  input  to  AIDA:  1) 
information  about  students,  the  learning  environment,  and  the 
tasks  to  be  mastered  (SET) ,  and  2)  a  specification  of  the  content 
material  to  be  mastered.  The  executive  component  maps  the 
relevant  spaces  (students,  environments,  task,  and  content)  into 
a  set  of  instructional  strategies.  The  instructional  delivery 
component  refers  to  the  process  of  interpreting  or  running  those 
strategies  and  is  the  ultimate  output  of  AIDA. 

Halff's  paper  continued  with  an  elaboration  of  each  of  the  AIDA 
components.  With  regard  to  the  content  component,  Halff  sees 
these  three  issues  as  problematic:  the  function,  the  structure, 
and  che  actual  representation  of  a  particular  content  area.  The 
content  must  be  represented  in  a  generative,  cognitive  form 
(behavioral  models  do  not  allow  complex  instructional  objectives 
to  De  represented  nor  do  they  allow  for  intermediate  structures 
that  support  skilled  performance)  ;  this  results  in  a  competence 
model  and  the  problem  of  identifying  the  mechanisms  whereby 
competence  is  achieved.  The  generative  requirement  implies  a 
grammar  for  instruction  and  the  separation  of  structure  (syntax) 
from  content  (semantics)  with  an  explicit  representation  of  the 
instructional  objectives  in  the  content  component.  We  hive,  as 
yet,  no  grammar  for  instructional  design.  Moreover,  there  is,  on 
Halff's  view,  no  uniform  method  for  analyzing  particular  skills. 

With  regard  to  the  SET  component,  Halff  argued  that  the 
relationship  to  the  content  component  is  crucial  and  problematic: 
in  some  situations,  the  two  components  are  functionally 
independent  but  in  others  they  are  closely  interdependent. 

The  instructional  strategies  component  is  comprised  of  procedures 
for  delivering  instruction  --  one  strategy  for  a  single 
instructional  objective.  The  representation  of  instructional 
strategies  is  another  crucial  issue  for  AIDA.  In  general,  there 
are  two  approaches,  according  to  Halff:  generative  and 
schematic.  The  generative  approaches  assume  the  existence  of 
instructional  primitives  and  procedures  for  manipulating  those 
primitives  into  strategies  (a  transformational  grammar  of  sorts) . 
Such  approaches  ter.d  to  be  best  tried  in  very  narrowly  confined 
domains,  as  we  lack  a  general  grammar  of  instruction.  Schematic 
approaches  are  based  on  configurable  black  boxes  or  transaction 
frames  which  can  be  adapted  to  a  wide  variety  of  settings. 
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Halff  also  s^es  serious  challenges  for  the  AIDA  executive 
component,  which  must  analyze  a  broad  range  of  content  and  SET 
information  and  produce  a  set  of  procedures  to  effectively  teach 
the  material.  Being  able  to  perform  the  required  transformations 
on  various  repiesanuat  ; or.s  into  forms  suitable  for  students, 
designers,  and  .arious  automated  devices  is  no  simple  task.  The 
executive  r: rses  content  into  objectives  and  uses  those  and  SET 
to  determ -.i.o  the  most  promising  strategy  to  pursue. 

Since  strategies  have  procedural  representations,  instruction  can 
be  delive  .ed  by  executing  the  strategy.  Taking  into  account  the 
level  of  Interaction  implied  in  the  procedure  is  a  serious 
consider  . cion.  Another  challenge  is  matching  the  power  of  the 
interpreter  to  the  level  of  abstraction  in  the  strategy 
specifications. 

Halff  stress’d  the  importance  of  evaluation  throughout  the 
instructional  p-ocess.  Three  levels  of  evaluation  were 
identified:  1;  >.  .alicy  control  (Am  I  doing  tl  2  process  right?), 

2)  formative  evaxuati-n  'Is  It  the  right  process?),  and  3) 
summative  evaluation  (Are  our  underlying  assumptions  right?) . 
Halff  indicated  that  it  was  important  to  isolate  the  level  of 
problems  and  to  identify  the  particular  AIDA  component  at  fauit, 

Halff  then  imagined  two  incarnations  of  AIDA,  one  in  an  Air  force 
Technical  Training  Center  (TTC)  and  another  called  CAI  'R  Us. 
Halff  indicated  in  his  remarks  that  the  TTC  example  represents 
the  road  not  taken,  although  his  previous  remarks  ana  paper 
indicate  he  believes  this  is  the  road  most  worth  pursuing.  The 
content  domain  for  this  AIDA  is  highly  focused  (e.g.,  radar 
maintenance) ,  the  SET  characteristics  are  simple  and 
straightforward,  instructional  strategies  can  be  formalized,  an 
executive  built  around  ready-made  plans  and  default  slots  can  be 
implemented,  and  design  and  delivery  can  be  automated  for  a 
variety  of  media.  In  short,  Halff  envisioned  an  aIDA  that 
embodied  instructional  design  theory  as  it  pertains  to  radar 
maintenance . 

The  CAI  'R  Us  example  presented  a  conception  of  AIDA  for 
knowledgeable  instructional  designers  that  cculd  be  applied  to 
many  domains  where  computer-based  instruction  is  being  developed. 
This  concept  was  aimed  at  providing  automated  implementation  of 
CAI  instructional  strategies.  To  accommodate  a  variety  of 
subject  domains  and  yet  minimize  the  number  of  representation 
schemes,  CAI  '  R  Us  used  production  systems  to  represent 
procedural  knowledge,  schema  to  represent  declarative  knowledge, 
and  qualitative  process  theory  to  represent  causal  knowledge. 

The  SET  component  of  this  AIDA  uses  a  simple,  fixed  vector  of 
features  The  strategies  are  collections  of  instructional 
schemata  --  configurable  and  specialized  to  different  levels  of 
abstraction.  The  executive  in  this  AIDA  is  a  top-down  course 
designer.  Delivery  of  instruction  is  restricted  to  CAI  systems 
with  a  middle  level  of  interactivity. 
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The  CAI  'R  Us  AIDA  concept  was  also  termed  an  'implementor',  in 
contrast  with  the  'embodier'  model  behind  the  TTC  model.  A  third 
kind  of  AIDA  was  sketched  briefly  by  Halff,  and  it  was  termed  a 
facilitator.  The  facilitator  offered  timely  reminders  and 
guidance  and  was  likened  to  Gagne's  concept  of  AIDA. 

Halff  identified  particular  design  and  research  issues  for  the 
two  kinds  of  AIDA  that  he  has  proposed.  The  TTC  AIDA  is  aimed  at 
SMEs  unskilled  in  instructional  design.  It  assumes  that  Training 
Development  Branches  (TDBs)  are  organized  along  narrowly  defined 
technical  areas.  It  assumes  that  classes  typically  occur  in 
small  lecture-discussion  settings  with  occasional  use  of  a 
variety  of  supplementary  media.  Key  design  issues  include  the 
development  of  specialized  curriculum  templates  appropriate  to 
the  particular  domain,  instructional  strategies  for  the 
particular  skills  involved,  and  content  representations 
sufficient  to  provide  material  for  the  curricula  and 
instructional  strategies.  A  critical  research  issue  is  an 
explanation  of  how  instruction  becomes  progressively  constrained 
as  content  becomes  progressively  specialized. 

The  CAI  'R  Us  AIDA  is  aimed  at  skilled  instructional  designers 
using  SMEs  as  clients/consultants.  This  AIDA  is  intended  to 
provide  a  means  to  build  curricula  for  a  variety  of  subject  areas 
and  to  implement  those  curricula  using  CAI.  Key  design  issues  in 
this  case  include  the  development  of  a  set  of  instructional 
strategies  defining  a  course  at  all  levels,  transaction  frames 
tailored  to  the  methods  and  media  available,  and  general 
knowledge  representation  mechanisms.  Research  issues  include 
defining  initial  target  domains,  determination  of  the  flexibility 
of  a  set  of  knowledge  representations,  and  the  assessment  of  the 
applicability  of  a  top-down,  hierarchically  dependent  development 
process . 


4.2.4  Merrill's  Cycle  2  Paper:  "An  AIDA  Concept1' 


Summary 

Merrill's  paper  and  remarks  were  intended  to  provide  coherence  to 
the  outline  for  AIDA  suggested  by  Merrill  and  tentatively  adopted 
at  the  December  AIDA  meeting.  The  paper  consists  of  an 
elaboration  of  the  attributes  and  range  of  values  associated  with 
each  of  the  AIDA  components. 

Within  the  information  component,  for  example,  Merrill  proposes 
that  these  attributes  are  pertinent  to  decision-making  involving 
the  audience:  role,  motivation,  familiarity,  mastery  level,  and 
ability.  Sample  values  for  the  role  attribute  are  consumer, 
supervisor,  technician,  and  problem  solver.  The  task 
subcomponent  of  the  information  component  introduced  the  notion 
of  enterprise  as  a  complex,  bounded,  integrated  meaningful  human 
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activity.  The  inadequacy  of  behavioral  objectives  or  goal  lists 
to  account  for  complex  learning  tasks  brought  the  need  for  this 
concept  to  light.  Seven  enterprise  classes  were  identified: 


denoting 


executing 


communicating  the  identity  or  describing  the 
form  or  structure  of  some  entity,  activity, 
or  process. 

performing  some  activity. 


evaluating 

designing 

manifesting 


—  judging  the  performance  of  an  activity. 

—  devising  a  novel  activity. 

—  making  a  process  sequence  evident 


interpreting  --  analyzing  the  cause  and  effect  relationships 

of  a  process. 


discovering  —  bringing  to  light  a  new  process. 


With  regard  to  the  content  component,  Merrill  indicated  that  he 
was  already  underway  with  this  work  as  part  of  a  Knowledge 
Acquisition  and  Analysis  System  being  developed  for  IBM.  Issues 
yet  to  be  addressed  include  the  rules  governing  necessary  and 
sufficient  associations  among  frames,  the  rules  governing  the 
propagation  of  information  from  frame  to  frame,  and  the  rules  to 
maintain  consistency  of  information. 

The  elaborated  frame  network  (EFN)  approach  was  chosen  because  it 
represented  a  compromise  between  semantic  networks  (too  low 
level)  and  frames  or  schemata.  The  nodes  in  the  EFN  are  frames. 

A  frame  is  elaborated  in  terms  of  its  links  to  other  frames. 

There  are  three  kinds  of  elaboration:  associations  with  other 
frames,  membership  in  an  abstraction  hierarchy,  and  components 
consisting  of  all  the  knowledge  and  skill  associated  with  a 
frame. 


The  AIDA  executive  takes  the  information  and  content  components 
and  prescribes  strategies  or  transactions  for  the  student.  Two 
functions  are  important  for  this  function:  prescriptions  and 
filters  for  knowledge  acquisition,  and  2)  selection  and 
configuration  of  transactions.  Transactions  are  real-time, 
give-and-take  exchanges  between  the  student  and  an  instructional 
system.  Transaction  shells  provide  the  structure  of  a 
transaction  identifying  the  parameters,  interactions,  content 
needed,  etc. 

Transactions  may  be  expository  or  inquisitory,  learner  or  system 
controlled,  tutorial  or  experiential,  and  capable  of  multi-media 
support.  Transactions  can  be  grouped  into  enterprise 
transactions,  association  transactions,  abstraction  transactions, 
and  component  transactions.  An  enterprise  transaction  is  the 
control  structure  for  an  entire  course  or  unit  of  instruction. 
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It  includes  a  primary  sequencing  strategy  (encyclopedic,  case 
study,  or  naturalistic)  and  a  secondary  sequencing  strategy 
(vertical,  temporal,  graded,  or  abstraction). 

A  critical  question  to  be  addressed  in  an  AIDA  is  to  determine 
how  to  relate  the  knowledge  base  to  various  transactions. 
Merrill  indicated  that  interface  issues  were  separate  from  the 
system  capabilities  that  he  had  been  sketching.  Merrill  does 
believe  that  a  system  can  be  made  accessible  to  novice 
instructional  designers  and  that  such  a  system  should  provide 
both  explanations  and  alternatives. 


4.2.5  Poison's  Cycle  2  Paper:  "AIDA  Cycle  2" 


Summary 

Poison's  approach  was  to  interview  a  specialist  from  a  Technical 
Training  Center  and  to  analyze  Friedman's  results  as  they  pertain 
to  AIDA.  Poison  reported  that  the  single  most  time  consuming 
task  in  developing  CBT  concerned  the  design  and  preparation  of 
visual  materials.  The  time-consuming  nature  of  this  task  and  the 
fact  that  it  is  not  consistently  well  executed  prompted  the  focus 
on  graphically  related  issues. 

The  five  problem  areas  identified  and  examined  by  Poison  were: 

1)  Use  of  visual  materials,  2)  Evaluation,  3)  Interactive 
Courseware  Design,  4)  Task  or  Instructional  Analysis,  and  5) 

Human  Computer  Interaction. 

Poison  cited  major  unresolved  issues  in  the  areas  of 
interactivity  and  human  computer  interaction.  In  CBT  settings, 
it  is  crucial  to  decide  who  (learner/system)  should  control  what 
portions  of  the  lesson,  when  particular  control  should  shift,  and 
how  control  should  be  made  effective.  To  construct  an  AIDA  which 
makes  these  decisions  would  require  formalizing  and  systematizing 
those  decision  procedures. 

Poison  also  suggested  using  a  cognitive  walkthrough  document  in 
order  to  make  decisions  about  the  future  development  of  AIDA. 

The  major  sections  of  this  document  are  as  follows: 

Initial  Task  Description 

Describe  the  task  from  the  point  of  view  of  the 
first-time  user. 

Interface  Start-Up  Description 

Describe  the  initial  state  of  the  interface  from 
user's  point  of  view. 
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Action  Sequence 

Make  a  numbered  list  of  the  atomic  actions  that 
the  user  must  perform  to  accomplish  the  task. 

Adjusting  the  Goals 

What  do  the  user's  current  goals  need  to  be  in 
order  to  keep  the  user  on  track? 

Discovering  the  Action 

Describe  the  next  atomic  action  that  the  user 
should  perform  (include  prompts  and  time-outs) . 

Performing  the  Action  and  Observing  its  Results 

Describe  how  the  user  will  physically  execute 
the  action  and  simulate  that  execution. 

Another  use  for  the  above  kind  of  questioning  would  be  in  a 
system  like  Gagne's  which  attempts  to  prompt  reasonably 
intelligent  subject  matter  experts  through  the  design  and 
development  of  instruction.  Asking  the  right  kind  of  question  at 
the  appropriate  time  provides  one  kind  of  intelligent 
facilitation. 


4.2.6  Tennyson's  Cycle  2  Paper (s):  "AIDA:  Framework 

Specifications  Document  for  an  Instructional  Systems 
Development  Expert  System" 


Summary 

Tennyson's  paper  argues  that  a  cognitively  updated,  fourth 
generation  of  ISD  (ISD-4)  provides  a  useful  framework  for  AIDA. 
Tennyson's  ISD-4  rejects  the  behavioral  bias  of  previous  ISD 
models  in  favor  of  cognitive  analysis  of  learning  tasks.  ISD-4 
also  rejects  the  linear  approach  to  instructional  design  and 
development. 

In  addition,  Tennyson  proposed  designing  a  system  not  just  for 
novice  instructional  designers,  because  novices  don't  remain 
novices  and  Dallman  had  indicated  a  need  to  accommodate  a  range 
of  users  at  the  December  meeting.  As  a  result,  Tennyson's  AIDA 
would  be  an  expert  system (s)  that  has  both  diagnostic  and 
prescriptive  functions  and  coaches  novices  but  advises 
experienced  users.  AIDA  would  employ  a  tutoring  capability  (ITS) 
aimed  at  improving  the  user's  knowledge  of  instructional  design. 
AIDA  would  have  a  definite  cognitive  bias  in  the  analysis  of 
learning  tasks  and  in  the  design  of  user  interfaces.  The  user 
could  enter  the  system  at  a  place  appropriate  to  the  individual 
needs  at  hand.  Instructional  training  could  occur  in  tandem  with 
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the  development  of  training  materials.  To  accomplish  such 
ambitious  results,  a  mainframe  with  remote  processing 
capabilities  would  be  required,  as  would  a  structured  query 
language  and  extensive  data  bases-  Tennyson  did  point  out  that 
maintaining  massive  data  bases  is  not  a  trivial  task. 

Although  Tennyson  barely  mentioned  his  Appendix  B  (authoring 
activities  knowledge  base  for  ISD-4) ,  the  substance  of  Tennyson's 
detailed  conception  of  AIDA  is  best  represented  in  that  appendix. 

The  left  column  of  the  appendix  represents  the  updated  ISD 
phases;  the  right  column  represents  the  associated  authoring 
activities,  which  would  presumably  represent  the  functional 
characteristics  of  an  AIDA  informed  by  Tennyson's  updated  ISD 
model . 

Tennyson  revised  this  paper  several  times.  In  the  revisions,  Tennyson 
dropped  the  notion  of  an  intelligent  tutoring  system  for  ISD  in 
favor  of  an  updated  ISD  process  with  an  intelligent  interface. 

The  revised  notional  structure  consisted  of  an  ISD  Expert 
supported  by  three  functional  components:  1)  Situational 
Evaluation/Diagnosis,  2)  Recommendations/Prescriptions,  and  3) 
Mini-Expert  to  Guide  Users.  The  intelligent  interface  contains 
the  ISD  Expert,  an  ISD  Model  Knowledge  Base,  the  Instructor's 
Model  of  ISD,  and  a  Content  Knowledge  Base  containing  previously 
developed  lesson  materials.  Complete  novices  are  coached  through 
the  system  by  the  ISD  Expert  which  makes  most  of  the  decisions. 

More  advanced  users  are  advised  through  the  system  with  users 
able  to  preempt  system  defaults  and  decisions. 

Tennyson  also  dropped  the  requirement  for  a  mainframe  and 
restated  that  requirement  in  the  form  of  a  need  for  an  open 
architecture  and  a  centralized,  large  capacity  database  with 
remote  access  via  personal  computers  or  workstations.  He 
estimated  that  it  might  take  five  years  and  $10  million  to 
complete  such  a  system. 


4.2.7  O'Neil's  Cycle  2  Critique 


Summary 

O'Neil's  purpose  was  to  suggest  what  AIDA  should  contain  and  what 
is  missing.  His  theme  was  that  there  was  more  on  the  AIDA  plate 
than  could  imaginably  be  funded.  Reducing  the  functionality  of  a 
first-cut  AIDA  while  maintaining  some  diversity  in  the  functional 
specifications  would  be  a  challenge. 

O'Neil  viewed  the  context  as  one  of  conceptualizing  and 
prototyping  a  strawman  system  and  then  testing  and  fixing  that 
system.  O'Neil  thought  the  inclusion  of  default  values  in  the 
first-cut  system  was  a  good  idea.  He  thought  an  explanation 
facility  should  be  added  to  Merrill's  transaction  shells.  A 
reference  library  has  yet  to  be  established  to  support  that 
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facility;  this  could  be  the  focus  of  a  6.1  basic  research  effort. 
O'Neil  viewed  Merrill's  AIDA  as  full  of  diversity  and  relatively 
expensive.  A  first-cut  AIDA  would  have  to  be  more  restricted 
than  Merrill  envisioned. 

O'Neil  said  that  Halff's  contrast  of  CAI  'R  Us  and  the  TTC  System 
was  quite  to  the  point.  Another  way  to  focus  AIDA  would  be  to 
aim  the  effort  at  tools  to  improve  the  contracting  effort. 

O'Neil  thought  Tennyson's  revised  cycle  2  paper  was  an 
excellent  conception  of  the  next  generation  of  ISD.  The  next 
generation  needed  to  consider  the  level  of  users  and  adjust  the 
ISD  model  to  the  user's  level  —  hereafter  dubbed  AISD  for 
adaptive  ISD. 

O'Neil  identified  two  primary  viewpoints  with  regard  to  cycle  2 
AIDA  conceptualizations:  1)  A  development  view  (Gagne,  Merrill, 
and  Tennyson) ,  and  2)  A  research  view  (Halff ,  Poison,  Friedman) . 
As  a  consequence,  O'Neil  stated  a  need  to  reach  closure  on 
several  of  these  issues:  1)  Whether  to  include  alternative 
configurations  of  AIDA,  2)  Whether  to  use  an  incremental 
development  model,  3)  How  to  prioritize  features,  and  4)  Whether 
to  make  an  embedded  AIDA  (perhaps  embedded  in  ATS,  for  example) . 

O'Neil  made  it  a  point  to  separate  development  and  delivery 
issues  in  contrast  to  Merrill.  He  reiterated  the  need  for  an 
explanation  facility.  O'Neil  also  indicated  the  need  to  update 
the  Friedman  paper.  If  the  focus  continued  to  be  on  procedural 
learning,  then  it  would  be  necessary  to  consolidate  relevant 
research  findings  in  one  place.  O'Neil  indicated  that  it  would 
be  desirable  to  include  some  emphasis  on  metacognitive  learning 
in  an  AIDA. 

O'Neil  indicated  that  AIDA  needed  an  author  management  system  and 
a  course  management  system.  Without  computer-managed  instruction 
there  would  be  no  benchmarks  to  determine  progress  and  there 
would  be  weak  Air  Force  support. 

The  AIDA  project  could  also  provide  sets  of  lessons  learned  to 
transition  to  the  field.  Answering  these  questions  would  go  a 
long  way  in  that  direction:  1)  Why  is  traditional  ISD  dead  in 
the  DoD?  2)  Why  didn't  the  Air  Force  adopt  ISS?  3)  What  success 
stories  for  authoring  systems  exist?  and  4)  What  are  the  cost 
drivers  and  tradeoffs  pertinent  to  authoring  systems? 

O'Neil  stressed  the  need  for  a  more  attractive  metaphor  to 
iuprove  the  image  of  AIDA  (Halff  suggested  La  Boheme) .  It  should 
be  noted  that  one  AIDA  related  SBIR  effort  does  involve  an 
explicit  attempt  to  elaborate  such  a  metaphor  in  terms  of  an 
instructional  designer's  work  environment.  The  book  metaphor 
behind  IDEAL  Bookshelf  was  viewed  as  generally  effective. 
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O'Neil  cited  the  need  to  address  procedural  learning  and 
metacognitive  skills,  the  need  to  automate  existing  paperwork, 
the  need  to  integrate  an  Al-based  simulation  authoring  tool,  and 
the  need  to  include  an  evaluation  tool  kit  in  AIDA.  O'Neil 
believes  that  CBT  authoring  will  be  team-based,  yet  he  did  not 
see  how  AIDA  would  handle  team  authoring;  this  underlined  his 
interest  in  an  author  management  system. 

O'Neil  maintained  that  AIDA  should  be  able  to  handle  existing 
media  and  be  able  to  accommodate  new  media.  Ke  also  claimed  that 
it  was  not  too  early  to  give  thought  to  what  the  user  interface 
should  look  like.  It  should  be  noted  that  the  SBIR  effort 
referred  to  earlier  is  also  providing  a  version  of  a  user 
interface.  In  addition,  evaluations  of  Merrill's  transaction 
shells  should  provide  further  input  data  to  user  interface 
decisions . 

O'Neil  thought  that  if  the  near  term  AIDA  were  to  be  a  research 
tool,  then  measurement  and  evaluation  issues  needed  to  be 
confronted  directly  and  soon.  More  data  gathering  would  have  to 
be  put  into  the  system  to  allow  it  to  function  as  a  research 
tool . 

O'Neil  addressed  2  issues  raised  in  Tennyson's  paper:  1)  A 
centralized  facility  for  sharing  courseware,  and  2)  A  database  of 
representative  users.  The  ATC/XPC  advisors  reacted  in  a  positive 
manner  to  these  suggestions,  although  they  are  most  likely 
expensive  options.  O'Nei  also  recommended  targeting  a 
particular  Air  Force  system  to  use  in  developing  instruction  with 
an  experimental  AIDA.  Dallman  made  a  similar  recommendation  in 
his  presentation  on  Day  1. 

O'Neil  concluded  with  a  list  of  eight  critical  issues  that  needed 
to  be  addressed  in  an  AIDA  system: 

1.  An  investment  strategy. 

2.  Benchmarks. 

3.  An  explanation  system. 

4 .  Prototype  tryouts  with  users . 

5.  An  R&D  agenda. 

6.  A  CMI  system. 

7.  Focus  on  procedural  and  metacognitive  skills. 

8.  State-of-the-art  assessments. 

By  way  of  summary,  O'Neil  has  recommended  that  we  narrow  the  AIDA 
focus,  but  that  we  do  not  make  it  so  narrow  to  exclude  issues 
pertaining  to  the  management  and  evaluation  of  courseware  design, 
development,  and  delivery.  Several  of  O'Neil's  final  eight 
suggestions  have  already  been  accepted  (e.g.,  #1  and  #5  above  are 
already  addressed  in  the  AFHRL/IDC  AIDA  Long  Range  Plan  and 
included  here  in  section  1.2). 
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4.2.8  Reigeluth 's  Cycle  2  Critique 


Summary 

Reigeluth  provided  a  much  different  perspective  on  how  to  proceed 
with  implementing  an  AIDA.  He  believed  that  a  first-cut  AIDA 
system  should  epitomize  the  long  range  AIDA.  As  a  consequence, 
it  becomes  more  important  to  articulate  the  grand  view  of  AIDA  as 
much  as  possible.  Reigeluth  argued  that  developing  an  epitome 
system  using  simplifying  conditions  to  enhance  success  and 
investment  in  all  parts  would  simplify  all  four  thrusts  of  the 
long  range  plan  and  provide  a  rational  development  model. 

Reigeluth  agreed  with  the  focus  on  intermediate  users  in  the 
first-cut  system.  He  would  further  restrict  the  subject  matter 
domain  from  electronics  maintenance  to  something  like  avionics 
maintenance.  He  also  agreed  with  the  decision  to  emphasize  an 
instructional  design  advising  system  first  and  worry  about 
automated  development  and  delivery  later. 

Reigeluth  disagreed  with  the  tentative  plan  to  have  the  starting 
input  to  a  first-cut  AIDA  be  well-formulated  learning 
capabilities.  He  preferred  to  start  with  the  epitome  of  input, 
such  as  a  cognitive  task  analysis  or  possibly  an  enterprise 
analysis. 

With  regard  to  the  individual  cycle  2  papers,  Reigeluth  made  the 
general  claim  that  all  the  authors  needed  to  epitomize  and  to 
simplify.  For  example,  with  regard  to  Gagne's  conception  of 
AIDA,  Reigeluth  recommended  omitting  elaborations  of  features 
pertaining  to  non-readers,  motivation,  and  instructor-based 
settings  as  a  way  of  incorporating  simplifying  conditions  in  the 
first-cut  AIDA. 

Reigeluth  argued  that  Merrill's  conception  of  AIDA  could  be 
simplified  in  a  number  of  ways  (e.g.,  restrict  the  audience, 
restrict  the  environment,  restrict  the  task,  restrict  the 
transactions  involved,  restrict  user  control,  etc.). 

Reigeluth  found  Halff's  Figure  1  (Merrill's  six  functional 
blocks)  thought  provoking  and  offered  a  complicated  extension. 
Reigeluth  argued  that  the  AIDA  epitome  should  include  nine  mini¬ 
experts,  but  for  highly  restricted  domains. 

While  his  line  of  reasoning  is  similar  to  Halff's,  Reigeluth  did 
point  out  some  differences  with  Halff.  With  regard  to  Halff’s 
three  dichotomies  (general  vs.  domain  specific,  black  box 
transactions  vs.  primitives,  and  general  vs.  detailed 
specifications) ,  Reigeluth  denied  the  inescapable  nature  of  th® 
dichotomies  and  recommended  a  compromise  combination  in  each 
case. 


44 


Reigeluth  warned  against  confusing  the  operationalization  of 
strategies  with  the  automation  and  instantiation  of  strategies. 
The  former  applies  to  particular  content  domains;  the  latter 
apply  to  particular  delivery  systems.  In  general,  Reigeluth 
liked  Halff's  focus  and  viewed  it  as  a  version  of  epitomizing. 

Reigeluth  found  many  useful  items  on  visual  materials  in  the 
Poison/Friedman  papers.  However,  he  thought  they  were  too 
advanced  to  be  incorporated  into  a  first-cut  AIDA  system. 

Reigeluth  questioned  whether  tutoring  should  be  a  primary  feature 
of  AIDA.  Reigeluth  indicated  that  it  was  not  clear  who 
Tennyson's  users  were.  Reigeluth  saw  AIDA  as  replacing,  not 
retraining,  instructional  designers.  The  idea  of  AIDA  was  to 
help  SMEs  develop  effective  courseware  without  resorting  to  the 
expense  of  human  expert  instructional  designers.  Reigeluth  would 
omit  the  content  knowledge  base  from  the  first-cut  system. 

Reigeluth  also  warned  against  attempting  to  break  new  ground  in 
all  areas.  For  example,  AIDA  should  not  try  to  break  new  ground 
in  the  areas  of  expert  system  methodologies. 
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4.3  SYNOPSES  OF  MEETINGS 


Five  technical  meetings  were  held  as  part  of  the  Phase  I  AIDA 
effort.  Hickey  taped  the  meetings  and  provided  transcripts  of 
key  discussions  to  participants.  Spector  compiled  progress 
reviews  distributed  after  each  meeting.  The  information 
contained  in  the  following  summaries  is  drawn  primarily  from 
those  reviews. 


4.3.1  August  Kick-off  Meetings 

The  kick-off  meeting  was  divided  into  three  parts  in  order  to 
accommodate  the  schedules  of  the  consultants.  In  the  first  part, 
the  history  of  the  AIDA  concept  was  reviewed,  Mei  Associates' 
statement  of  work  was  reviewed,  and  an  agenda  for  the  second  part 
of  the  kick-off  was  developed.  It  was  decided  to  divide  the  task 
into  two  cycles  and  to  use  two  critiquers.  Gagne  suggested 
specific  assignments  for  the  consultants  (see  Appendix  B  for  task 
assignments) . 

In  the  second  part  of  the  kick-off  meeting,  Merrill  wanted  AFHRL 
to  provide  more  detailed  and  specific  guidance  about  AIDA. 

Newcomb  did  provide  additional  detail  but  argued  that  since  this 
was  a  6.2  exploratory  development  effort  that  it  was  not 
appropriate  to  provide  a  requirements  analysis  and  functional 
specifications;  indeed,  that  was  primarily  what  was  sought  from 
the  consultants.  Halff  argued  that  very  few  general 
instructional  design  principles  could  be  incorporated  into  a 
general  purpose  AIDA.  As  a  consequence,  he  maintained  that  there 
would  have  to  be  many  AIDAs .  Tennyson  saw  a  need  to  establish  an 
underlying  philosophy  and  set  of  assumptions  before  continuing 
with  design  requirements  for  AIDA.  Gagne's  task  assignments  were 
revised  to  reflect  a  shift  in  emphasis  from  theory  development  to 
the  design  of  a  prototype  AIDA. 

In  the  third  part  of  the  kick-off  meeting,  Gagne  asked  AFHRL  to 
identify  the  products  for  Task  6.  Newcomb  indicated  that  the 
product  was  to  be  a  description  of  the  content  of  a  potential 
prototype  of  AIDA;  no  software  production  was  involved  in  Task  6. 
Poison  wanted  all  consultants  to  respond  to  some  common  task  so 
that  there  would  be  some  basis  for  performing  a  synthesis  of  the 
responses.  The  degree  of  intelligence  to  be  incorporated  into 
AIDA  was  also  identified  as  an  issue  in  need  of  further  analysis. 


4.3.2  October  Presentation  Meeting 

The  October  meeting  featured  papers  submitted  by  the  five 
consultants  and  a  needs  assessment  report  by  Muraida  and  Spector. 
Areas  of  consensus  and  disagreement  were  identified  (see  section 

4.3.3  below).  All  agreed  that  a  key  issue  is  determining  the 
extent  to  which  instructional  design  expertise  can  be  automated. 
To  make  this  determination,  several  instructional  mini-advisors 
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will  have  to  be  built  and  issues  concerning  the  tractability  of 
the  instructional  design  space  will  have  to  be  resolved. 
Justifying  and  elaborating  specific  instructional  design 
principles  and  identifying  appropriate  knowledge  representation 
schemes  were  also  considered  important  issues.  There  was  also 
general  agreement  that  cognitive  psychology  had  a  great  deal  to 
contribute  to  AIDA  and  that  contextual  knowledge  required  further 
analysis . 

The  principal  points  of  difference  concern  the  degree  to  which 
instructional  design  can  be  shifted  from  humans  to  computers,  the 
usefulness  of  existing  ISD  procedures,  and  the  extent  to  which 
instructional  design  principles  are  context  bound. 


4.3.3  December  Consolidation  Meeting 

The  December  AIDA  meeting  featured  a  review  of  the  cycle  1  papers 
by  Muraida  and  Spector,  critiques  of  those  papers  by  O'Neil  and 
Reigeluth,  and  planning  for  the  cycle  2.  The  cycle  1  review 
indicated  the  following  areas  of  agreement  and  disagreement: 


Areas  of  Agreement 

1.  There  is  a  lack  of  instructional  design  expertise  in  the  Air 
Force.  Tasks  requiring  training  are  becoming  increasingly 
complex  due  to  sophisticated  weapons  systems.  Training 
environments  are  becoming  increasingly  complex  due  to 
advanced  interactive  technologies.  As  a  consequence,  the 
need  for  instructional  design  expertise  is  becoming 
increasingly  acute. 

2.  Instructional  design  should  reflect  a  cognitive  perspective; 
behaviorism  is  not  adequate  in  providing  an  explanation  of 
complex  learning,  such  as  language  acquisition. 

3.  Knowledge  of  cognitive  structures  and  operations  is 
incomplete.  Cognitive  models  are  proposed  as  tentative 
guides  to  understanding  complex  mental  structures.  Exactly 
how  information  is  encoded  and  transformed  in  the  mind  is 
not  known.  That  coding  and  transformation  occur  is 
undisputed.  The  details  of  the  limitations  of  short  term 
memory  are  not  known,  but  that  there  are  limitations  is  well 
established . 

4.  There  are  several  high  level  instructional  principles  that 
have  been  derived  from  the  cognitive  sciences.  Kintsch's  10 
principles  of  instructional  theory  represent  a  good  sample. 
Merrill's  12  pedagogy  rules  and  Gagne's  12  instructional 
principles  indicate  much  agreement  and  overlap  at  this 
level.  There  is  also  agreement  that  the  detailed  micro¬ 
principles  have  yet  to  be  established  but  are  well  worth 
pursuing . 
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5. 


There  are  other  areas  of  agreement  with  regard  to  the 
influences  of  cognitive  learning  theory  on  instructional 
design.  For  example,  the  situated  and  integrated  nature  of 
human  activities  is  recognized  as  an  important  consideration 
in  designing  instruction.  Poison's  situational  models  and 
Merrill's  enterprises  reflect  this  view  directly. 

The  emphasis  on  integrating  new  knowledge  with  previously 
acquired  knowledge  also  reflects  agreement  in  this  area. 
Another  area  of  agreement  in  this  area  is  that  the 
organization  of  knowledge  in  memory  is  the  key  to  recall, 
retention,  and  transfer.  As  a  consequence,  instructional 
design  should  facilitate  the  organization  of  new  knowledge 
in  memory. 

In  addition,  the  distinction  between  declarative  and 
procedural  knowledge  is  fairly  clear  and  basic.  A  third 
type  of  knowledge,  contextual  knowledge  is  also  widely 
accepted. 

6.  Automating  course  design  and  development  can  potentially 
improve  quality  and/or  reduce  costs.  Automation  of 
instructional  design  expertise  should  contribute  to 
consistent  and  possibly  improved  quality.  Automation  of  the 
production  of  materials  in  accordance  with  that  advice 
should  reduce  costs. 

7.  Developing  an  experimental  prototype  is  needed  in  order  to 
make  real  progress  with  AIDA  related  issues. 


Areas  of  Difference 

1.  The  extent  to  which  instructional  design  expertise  can  be 
automated  is  in  dispute;  there  is  consensus  that  this 
dispute  is  best  settled  by  experimental  evidence. 

2.  The  usefulness  of  a  comprehensive  theoretical  basis  for 
instructional  design  is  not  completely  clear;  some  believe 
that  experiments  should  aim  toward  justifying  an  integrated 
instructional  theory  (Merrill,  Poison) ;  some  believe  that 
intuitions  of  successful  instructional  designers  are  a 
sufficient  foundation  for  proceeding  (Reigeluth) . 

3.  The  transference  of  cognitive  skills  outside  the  domain  in 
which  they  were  acquired  is  in  dispute;  some  believe  that 
cognition  is  always  situated  (Halff) ;  some  believe  that  some 
cognitive  skills  are  generalizable  beyond  the  situation 
(Merrill,  Tennyson,  Gagne). 
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4.  The  extent  of  the  influence  of  cultural  factors  is  in 
dispute?  most  believe  that  cultural  factors  play  a 
significant  role  in  learning  (Halff,  Poison,  Tennyson,); 
some  believe  that  the  resulting  pragmatics  of  cultural 
influences  cannot  be  captured  in  a  computer  program  (Halff) . 

5.  The  usefulness  of  ISD  is  in  dispute;  some  believe  ISD  is 
invaluable  (O'Neil) ;  some  believe  it  needs  modification  to 
be  useful  (Tennyson,  Gagne) ;  some  believe  it  can  be 
disregarded  (Merrill) ;  some  believe  it  is  fundamentally 
mistaken  (Halff) . 

O'Neil  viewed  the  AIDA  effort  as  ambitious  but  worthwhile. 

Desired  AIDA  capabilities  would  have  to  be  prioritized  due  to 
budgetary  constraints.  He  stressed  the  importance  of  assessment 
and  evaluation,  identified  a  variety  of  addicional  capabilities 
tc  consider,  indicated  several  ways  that  AIDA  could  be  made 
intelligent,  and  presented  three  sets  of  issues  for  AIDA  in  the 
area  of  project  management,  technology,  and  courseware  authoring 
management. 

Reigeluth  argued  that  the  AIDA  effort  should  be  based  on  an 
inductive  approach  since  useful  prescriptive  instructional 
principles  cannot  be  derived  from  existing  theories  of  learning. 
Reigeluth  supported  the  rapid  prototyping  of  an  initial  system 
based  on  first  generation  instructional  design  principles. 

The  current  task  was  narrowed  to  focus  on  the  following:  1) 
target  the  domain  of  apprentice  maintenance  training,  2)  target 
novice  instructional  designers,  3)  build  in  as  much  intelligence 
as  possible,  and  4)  automate  as  much  of  the  production  ~f 
instruction  as  possible.  Assignments  for  cycle  2  were  made  based 
on  a  conceptual  organization  provided  by  Merrill. 


4 .3. 4  February  Presentation  Meeting 

The  February  meeting  featured  presentations  of  papers  submitted 
by  the  consultants  and  planning  for  the  remainder  of  this  task. 

Tasks  for  the  April  meeting  were  established.  Four  critiquers 
were  tasked  in  the  April  meeting:  Dallman,  Marlino,  O'Neil,  and 
Reigeluth.  Dallman 's  task  was  to  select  three  maintenance 
training  applications  from  the  Lowry  TTC ,  one  to  include  a  CBT 
example,  for  inclusion  in  a  preliminary  data  base  of  training 
samples.  The  samples  should  include  appropriate  documentation. 
Dallman  was  asked  to  bring  at  least  one  CBT  course/lesson  to 
demonstrate  at  the  April  meeting. 

Marlino  agreed  to  evaluate  IDIOM  with  regard  to  lessons  learned, 
features  and  characteristics  to  include  in  AIDA,  and  things  to 
avoid.  She  also  agreed  to  provide  a  demonstration  of  IDIOM'S 
capabilities  at  the  April  meeting.  Merrill  also  agreed  to 
demonstrate  the  naming-the-parts  transaction  shell. 
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This  meeting  produced  six  outstanding  papers  related  to  issues  in 
automating  instructional  design  and  development.  There  was  still 
a  general  core  of  consensus  among  the  consultants  about  issues 
and  directions,  although  significant  differences  did  exist  (e.g., 
whether  to  target  novices  or  experienced  designers;  design  a 
general  purpose  system  or  design  a  narrowly  focused  system; 
provide  dynamic  expert  advice,  constructive  but  inexpert 
guidelines,  or  coach  novice  users;  assist  with  just  design,  or 
design  and  development;  etc.).  There  continued  to  be  consensus 
on  the  need  for  a  cognitive  learning  task  analysis,  although  its 
relationship  to  a  behavioral  job  task  analysis  was  not  clear 
(i.e.,  Should  the  behavioral  job  task  analysis  provide  more  than 
performance  criteria  for  input  to  the  design  of  instruction?  If 
so,  what  is  the  form  of  that  input  and  how  should/could  it  be 
used? )  . 

Halff  proposed  three  levels  of  AIDAs:  1)  AIDA  an  a  facilitator 
(an  earth  satellite/Gagne's  system),  2)  a  highly  focused  AIDA  in 
the  TTC  (a  lunar  satellite/Half f ' s  preference,  or  3)  a  general 
purpose  AIDA  (Merrill's  moonshot/CAI  'R  Us).  Halff  proposed 
developing  an  investment  strategy  for  each  path  and 
pursuing  coordinated  efforts  along  each  path.  The  coordination 
could  come  in  the  form  of  tools  developed  at  one  level  to  provide 
useful  inputs  to  the  next  level.  The  remainder  of  Halff' s 
critigue  was  made  in  terms  of  these  three  levels. 

Halff  described  Gagne's  AIDA  as  a  system  to  provide  constructive 
guidance  to  instructional  designers  and  developers.  It  was 
definitely  a  level  1  system.  It  could  be  considered  an 
intelligent  job  aid  for  instructional  designers/developers.  It 
would  be  an  ID-1  system  in  terms  of  theory.  It  would  require  the 
developer  to  perform  most  of  the  course  implementation.  The 
advantage  of  doing  work  at  level  1  would  be  to  provide  a  body  of 
consistently  good  instruction  (especially  good  computer-based 
instruction).  Tennyson's  proposal  to  have  a  content  knowledge 
base  (previously  developed  courseware)  can  also  be  construed  as  a 
level  1  pursuit  (Tennyson  obviously  proposed  doing  much  more) . 

Halff  described  Merrill's  AIDA  as  a  level  3  system.  It  would 
have  a  wide  area  of  applicability  and  the  system  would  do  much  of 
the  designing  and  developing  of  tne  instruction.  Halff  believes 
that  it  is  too  early  to  commit  great  resources  to  level  3, 
although  a  level  3  system  is  a  reasonable  long  range  goal.  Level 
3  systems  are  committed  to  ID-2,  as  are  level  2  systems. 

Halff  sees  level  2  systems  as  focused  on  very  particular  domains 
(e.g.,  radar  maintenance)  and  providing  a  body  of  good 
instructional  strategies  to  level  3. 

Merrill  responded  by  saying  that  there  were  already  numerous 
attempts  at  level  1  with  very  questionable  results.  Tennyson, 
Newcomb,  Ellis,  and  others  questioned  Merrill's  conclusions  about 
the  generally  poor  quality  of  most  CBT.  There  was  apparent 
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agreement  that  collecting  the  data  was  important  and  needed; 
Tennyson' s  content  knowledge  base  could  be  conceived  as  a 
repository  for  those  data.  Merrill,  though,  did  not  seem  anxious 
to  revisit  level  1  systems  (including  SOCPATES,  IDIOM,  Arthur 
Andersen's  Method  E,  IBM's  system,  Courseware's  system,  etc.) . 

Poison  wanted  to  know  if  the  level  1  user  should  be  a  novice  or 
an  experienced  user.  Halff's  response  was  to  use  an  expert  if 
your  goal  is  to  get  a  body  of  good  instruction  developed  to  use 
in  the  other  levels.  However,  Gagne's  level  1  system  is  aimed  at 
a  relatively  inexperienced  instructional  designer,  as  are 
SOCRATES  and  some  of  the  other  level  1  systems. 

Haiff  insisted  that  developing  a  tool  at  level  1  to  collect  good 
instruction  for  some  fairly  focused  area  would  make  it  obvious 
how  to  construct  ID-2  transaction  frames.  He  said  that  we  should 
become  good  natural  scientists  and  collect  some  samples. 

Merrill  proposed  that  Halff's  three  levels  looked  at  AIDA  in 
terms  of  tool  building.  That  was  just  one  dimension.  We  could 
look  at  AIDA  in  terms  of  instructional  design  theory  (ID-1  and 
ID-2)  or  in  terms  of  the  targeted  user  (novice,  experienced, 
expert  instructional  designer) .  Merrill  insisted  on  the  view 
that  we  should  not  be  involved  in  automating  the  past  (ID-1)  and 
that  we  needed  to  attend  to  advanced  technologies  available  and 
their  peculiar  characteristics.  AIDA  should  confront  the  issues 
of  interaction,  multi-media,  integration,  sequencing,  etc.  raised 
by  these  new  technologies. 

Merrill  indicated  that  it  was  possible  to  build  sample 
transaction  shells  as  a  stand-alone  set,  without  implementing  all 
of  his  system.  Merrill's  modification  of  level  1  would  be  to 
construct  such  shells,  test  them,  and  collect  some  useful  data. 

The  idea  occurred  to  Newcomb  that  Merrill's  transaction  shell  for 
naming  the  parts  could  be  used  in  a  stand-alone  study  to 
determine  how  much  development  time  could  be  saved  by  the  use  of 
such  shells.  Merrill  has  another  shell  for  teaching  checklist 
procedures  that  subsumes  the  "naming"  transaction  shell;  this 
shell  might  provide  the  means  to  carry  out  a  more  meaningful 
study . 


Gagr.e  indicated  that  we  needed  to  pick  a  more  serious  shell  than 
naming  the  parts,  as  it  was  too  easy.  He  suggested  that  we  try 
developing  a  shell  for  teaching  procedures.  Gagne's  suggestion 
met  with  much  approval  on  two  counts;  1)  We  needed  to  establish 
priorities  for  the  continuation  of  the  project,  and  2)  We  had 
a ' ready  agreed  to  focus  on  maintenance  training  in  which 
procedural  knowledge  would  play  a  large  role. 

Haiff  and  Merrill  seem  in  agreement  that  a  transaction  generator 
guidance  system  to  automate  instructional  design  and  development 
will  be  a  critical  component  of  AIDA.  Haiff  believes  it  will 
have  to  remain  focused  on  a  particular  domain.  Merrill  believes 
much  will  be  general  izable  across  domains. 


51 


Poison  saw  Merrill's  system  as  talcing  a  vertical  cut  through  the 
five  problems  uncovered  in  her  interview  with  Dallman  (graphics, 
evaluation,  interactivity,  task/ instructional  analysis,  and 
human-computer  interfaces),  while  Friedman's  paper  took  a 
horizontal  cut  at  the  problems  by  addressing  the  use  of  graphics 
in  all  domains.  She  admitted  that  the  vertical  but  focused 
approach  might  be  the  most  productive. 

Tennyson  also  proposed  three  different  levels  of  systems.  He 
agreed  with  Halff  that  ID  lacked  data;  his  level  1  system  would 
focus  on  developing  shells,  studying  user  reactions,  determining 
standards  for  shells,  and  experimenting  with  shells.  He,  like 
Halff,  thought  level  1  users  would  need  to  be  experts. 

Tennyson' s  level  2  involves  adding  diagnosis  so  that 
prescriptions  can  be  added  to  more  sophisticated  networks  of 
integrated  shells.  His  level  3  would  be  more  fully  automated  and 
accessible  to  novice  users. 

Halff  said  that  his  three  levels  did  not  directly  correspond  to 
Tennyson's.  Halff  claimed  that  Tennyson's  scheme  involved  ever 
larger  chunks  of  curriculum  at  each  level,  while  his  own  allowed 
all  curricular  structures  at  all  three  levels.  Halff  also 
claimed  that  artifacts  could  be  collected  at  all  levels.  The 
curricular  chunking  issue  does  not  seem  a  serious  differentiator 
between  Halff  and  Tennyson.  However,  there  appears  to  be  no 
place  in  Tennyson's  three  levels  for  Halff' s  level  2  system 
focused  on  a  TTC . 

Gagne  asserted  that  AIDA  should  be  primarily  an  advisor. 
Automating  instructional  development  was  not  likely  to  work, 
although  it  might  be  worth  a  try.  Gagne  insisted  that  we  needed 
to  design  for  the  five  basic  mental  capabilities,  and  the  system 
should  offer  advice,  examples,  and  alternatives. 

There  was  a  discussion  about  the  fundamental  categories  of 
capabilities  to  be  dealt  with  in  an  AIDA,  i.e.,  what  is  to  be 
learned.  Merrill  favors  denoting,  executing,  evaluating,  etc. 
Gagne  favors  concepts,  verbal  information,  rules,  etc.  Merrill 
agreed  that  some  of  his  shells  might  apply  to  larger  course 
units--more  comprehensive  capabilities--and  not  be  the  correct 
primitive  level  1  shells.  There  was  continued  verbal  dispute 
between  Gagne  and  Merrill  about  what  is  to  be  learned,  but  there 
appears  to  be  more  similarity  than  disagreement  (denotation  = 
label  information;  classification  =  concepts;  execution  = 
procedures;  etc.).  Merrill  admitted  that  Gagne's  five 
capabilities  must  be  accounted  for  at  a  basic  level  in  any 
system . 

Poison  remarked  that  a  critical  difference  between  Gagne  and 
Merrill  involved  how  much  control  over  the  design  of  instruction 
should  be  given  to  AIDA.  Gagne  wants  the  human  designer  heavily 
involved  in  the  planning  and  decision-making.  Merrill  would 
provide  much  control  for  the  system,  but  he  characterized  his 
AIDA  as  an  instructional  spreadsheet. 
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There  was  agreement  that  many  important  research  issues  remain 
unsolved  and  there  is  a  need  to  begin  building  a  large  data  base 
of  instructional  artifacts  and  associated  evaluations.  There  was 
agreement  that  AFHRL  needs  to  focus  and  prioritize  its  AIDA 
related  efforts  before  continuing  to  the  next  task.  A  long  range 
planning  document  was  suggested  for  this  purpose.  Since  AFHRL 
does  have  such  a  plan,  it  has  been  included  in  this  final  report 
(see  section  1.2). 

The  proposal  to  use  Merrill's  naming  transaction  shells  in  an 
evaluation  study  at  several  Air  Force  locations  will  be  pursued 
by  AFHRL/IDC  independently  of  this  task  or  the  Phase  II  task. 

It  was  agreed  that  O'Neil  and  Reigeluth  would  be  asked  to  analyze 
and  constructively  evaluate  the  concept  and  functionality  of 
AIDA,  the  long-range  plan,  and  the  associated  investment  strategy 
in  a  way  that  makes  a  smooth  transition  to  the  final  report  of 
this  task  and  the  beginning  of  the  next  task. 


4.3.5  April  Consolidation  Meeting 

This  meeting  featured  demonstrations  of  three  authoring 
systems/environments  by  Dallman,  Marlino,  and  Merrill;  critiques 
of  the  cycle  2  papers  by  O'Neil  and  Reigeluth;  a  review  of  ATC's 
ISD  procedures  by  Clemons;  a  review  of  ARI  projects  by  Seidel; 
and  the  completion  of  O'Neil's  AIDA  capabilities  survey. 


Marlino' s  IDIOM  Demonstration 


Dr.  Mary  Marlino  demonstrated  the  capabilities  of  IDIOM.  She 
began  her  discussion  by  identifying  three  levels  in  IDIOM:  1)  a 
broad  database,  2)  a  set  of  tools,  and  3)  a  low  level 
instructional  design  advisor.  She  indicated  that  IDIOM'S 
strength  was  the  breadth  of  its  database.  IDIOM'S  target 
audience  is  fairly  general,  containing  both  novice  and 
intermediate  instructional  designers.  IDIOM  is  an  advising 
system  developed  for  the  domain  of  sales  training.  IDIOM 
supports  both  procedural  and  declarative  knowledge  types.  Its 
instructional  design  theory  is  probably  first  generation.  One 
can  start  anywhere  in  this  easily  accessible  system.  IDIOM  has 
acquired  a  new  user-interface  and  been  renamed  IDEAL  Bookshelf. 

IDEAL  Bookshelf  is  a  menu-/button-driven,  hyper-media  type  system 
implemented  on  the  Macintosh  microcomputer.  The  opening  menu 
contains  the  following  choices: 
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IDEAL  BOOKSHELF  TABLE  OF  CONTENTS 


PRELIMINARY  RESEARCH 
ANALYZE 

DESIGN 

DEVELOP 

OVERVIEW 

HELP 

MANAGE 

PRODUCE 

IMPLEMENT 

EVALUATE 

BASIC  TOOLSET 

TOOL  CATALOG 

PROCESS  GUIDELINES 

To  facilitate  use  of  the  system  there  are  help  and  reference 
cards  which  comprise  a  database  resource  library.  The  help  that 
is  available  at  the  top  level  table  of  contents  (see  above) 
consists  of  pull-down  menus  with  the  following  items: 

WHAT  IS  IT? 

WHY  IT'S  DONE 

WHAT  YOU  GET 

JUDGING  SUCCESS 

HOW  TO  DO  IT 

These  same  items  are  provided  for  virtually  any  item  in  IDEAL. 

For  example,  one  can  use  a  submenu  list  under  ANALYZE  to  find  out 
what  Mager's  Goal  Analysis  Tool  is  and  how  to  use  it. 

The  level  of  detail  behind  the  opening  table  of  contents  is 
relatively  sophisticated.  For  example,  DESIGN  has  the  following 
items : 


SPECIFY  OBJECTIVES 
SEQUENCE  OBJECTIVES 
DETERMINE  STRATEGY 
SELECT  MEDIA 
PREPARE  ASSESSMENT 

A  number  of  tools  for  assisting  with  a  particular  item  are 
available  and  indicated  with  an  asterisk.  The  instructional 
designer  tool  is  as  interactive  and  as  prescriptive  as  any  in 
IDEAL  Bookshelf.  It  allows  the  creation  of  new  instruction  or 
the  modification  of  existing  instruction.  It  was  noted  that  the 
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system  was  not  fully  integrated  in  that  objectives  indicated  in 
SPECIFY  OBJECTIFIES  did  not  carry  forward  to  other  parts  of  the 
system.  Marlino  indicated  that  Apple  Computer  was  currently 
working  on  complete  integration  and  on  the  PROCESS  GUIDELINES. 

Objectives  are  classified  by  performance  (e.g.,  find,  use, 
remember)  and  by  content  (fact,  concept,  procedure,  and 
principle)  as  suggested  in  Merrill's  Component  Display  Theory. 
Gagne's  nine  events  of  instruction  were  also  a  strong  influence 
on  the  design  of  IDEAL  Bookshelf.  For  the  first  event,  gain 
attention,  the  user  is  offered  a  variety  of  means  to  accomplish 
it,  plus  a  rationale  for  doing  so. 

IDEAL  Bookshelf  is  not  a  lock-step  system,  however.  The  user  can 
skip  any  item.  The  advice  offered  users  generally  comes  in  the 
form  of  a  generality  followed  by  one  or  more  instances.  Marlino 
classified  IDEAL  as  a  facilitator  (Halff's  first  level)  with  very 
high  user  control.  There  was  insufficient  data  to  determine 
whether  or  exactly  what  using  IDEAL  contributed  to  the 
development  of  courseware. 

The  book  metaphor  is  the  guiding  and  operative  metaphor  behind 
luEAL  Bookshelf,  e.g.,  the  table  of  contents  through  which  one 
gains  access  to  the  system.  As  one  digs  deeper  into  the  system, 
one  finds  some  templates  for  course  documentation  provided  in 
MacWrite.  The  output  of  the  system  consists  of  nicely  formatted 
course  materials.  IDEAL  Bookshelf  is  not  a  delivery  system;  it 
is  a  design  and  development  environment  complete  with  a  number  of 
tools.  IDEAL  Bookshelf  is  currently  being  Beta-tested  by  several 
companies,  including  Federal  Express  and  Kodak. 

Dallman  and  Ellis  thought  that  SMEs  who  were  novice  instructional 
designers  would  not  be  able  to  use  IDEAL  immediately,  because, 
for  example,  they  would  not  know  the  difference  between  'find' 
and  'use'  or  between  'verbal  information'  and  'cognitive 
strategy'.  Reigeluth  characterized  IDEAL  as  a  tutor  and 
indicated  that  AIDA  might  be  less  of  a  tutor.  Matoon  saw  the 
lack  of  integration  of  what's  entered  into  the  system  as  a  major 
limitation?  most  agreed. 


Dallman *s  TTC  Demonstrations 

Dallman  began  his  demonstration  by  stating  his  objective:  to 
identify  three  maintenance  courses  which  would  comprise  a 
training  database  for  use  in  the  AIDA  program.  He  used 
four  selection  criteria:  1)  Developed/maintained  by  the  Air 
Force,  2)  Reasonable  examples  of  first  generation  instructional 
design  principles,  3)  Range  of  technical,  operator,  and  soft 
skills  addressed  by  the  instruction,  and  4)  Development  to  occur 
over  the  next  few  years. 
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Dallman  then  reviewed  common  problem  areas  in  CBT  systems  design: 
1)  Front-end  analysis,  2)  Content  analysis,  3)  Human-computer 
interaction,  4)  Limited  experience  base,  and  5)  Complexities  of 
interactive  courseware  design.  Dallman  focused  on  problems  in 
the  last  category  and  further  elaborated  that  category  as 
follows:  1)  Wrong  or  inappropriate  objectives  chosen  due  to  the 

use  of  canned  words,  2)  Compiled  knowledge  not  dealt  with 
properly  (failure  to  use  bite-sized  chunks  and  missing 
declarative/procedural  knowledge  in  course  materials) ,  3) 
Misconceptions  about  interactions  (e.g.,  failure  to  require 
application  of  domain  knowledge,  failure  to  foster  "deep” 
processing  capabilities,  etc.),  4)  Poor  or  inappropriate  use  of 
graphics  (e.g.,  creating  dissonance,  failure  to  encourage  dual 
encoding,  etc.),  5)  Failure  to  perform  formative  evaluation  with 
an  instructional  diagnosis,  6)  Poor  strategic  and  tactical 
instructional  design  decision  making  (e.g.,  overuse  of  tutorials; 
avoidance  of  simulations,  modeling,  and  gaming;  and  overuse  of 
instructional  transactions  that  only  required  recognition,  7) 
Inappropriate  use  of  learner  control,  and  8)  Inappropriate 
mastery  assessment  without  any  adaptive  testing  (see  #1  in  this 
list)  . 

Dallman  then  showed  a  film  clip  of  the  three  training  efforts  he 
was  recommending  for  an  initial  AIDA  database.  The  group 
generally  felt  that  the  first  effort.  Apprentice  F-16/A-10 
Avionics  Test  Station  and  Component  Specialist,  was  an  example  of 
excellent  CBT  and  IVD  using  the  QUEST  authoring  system.  Dallman 
had  the  other  QUEST  system  (no  IVD  involved)  running  on  a  Z-248 
and  presented  its  main  features. 

The  other  two  examples  involved  a  second  course  developed  using 
QUEST  and  a  course  developed  using  AIS-II,  both  on  Z-248 
microcomputers.  The  two  courses  were  generally  regarded  as  less 
interactive  than  the  first,  although  they  contained  many  useful 
examples  worth  considering  for  an  AIDA  database.  One  course  is 
for  the  apprentice  space  systems  operations  specialist,  and  the 
other  is  for  the  material  storage  and  distribution  exportable 
training  system.  The  material  storage  and  distribution  course 
represented  a  future  direction  of  some  training  —  toward  more 
distance  and  on-job-site  learning.  Dallman  classified  the 
content  as  in  the  soft  skills  domain  and  indicated  that 
declarative  and  procedural  hierarchies  were  developed.  Course 
authors  were  sensitized  to  information  processing  issues  (e.g., 
short-term  memory  limitations,  information  chunking,  the  role  of 
automaticity  in  compilation,  and  repair  theory) .  Extensive  use 
is  made  of  dual  encoding  and  visuals  for  motivation.  An  attempt 
to  implement  adaptive  testing  has  also  been  made.  Dallman  noted 
that  courseware  with  QUEST  was  requiring  about  350  hours  of 
development  per  hour  of  instruction  (not  counting  supervisory 
hours) ;  graphics  production  was  particularly  time-consuming. 
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The  apprentice  space  systems  operations  specialist  courseware 
involved  satellite  operations  personnel  and  IVD.  It  represents 
an  Air  Force  production  effort  made  by  personnel  with  no 
background  in  interactive  courseware  development.  Dallman  said 
there  are  various  style  guides  for  producing  courseware  pertinent 
to  particular  domains. 


Reactions  to  the  Demonstrations 

Following  the  IDIOM  and  TTC  demonstrations,  Halff  posed  seven 
questions  to  consider  in  evaluating  courseware  needs: 

1.  What  is/ ought  to  be  taught? 

2 .  How  should  it  be  taught? 

3 .  What  is  the  development  context?  What  is  the  corporate 
knowledge?  What  are  the  individual  competencies? 

4.  What  is  actually  developed? 

5.  What  accounts  for  the  differences  in  1  through  4? 

6.  How  do  we  implement  fixes  to  the  system? 

7.  If  one  had  to  build  AIDA  for  all  the  systems  involved  in  the 
demonstrations,  then  what  would  it  be  like?  How  would  AIDA 
work? 

Gagne  indicated  that  all  the  AIDAS  would  look  alike.  Halff 
disagreed  because  he  thought  the  development  context  would  be  a 
significant  and  major  determinant  of  system  requirements. 
Reigeluth  pointed  out  that,  regardless  of  the  dispute  between 
Gagne  and  Halff,  there  was  too  much  complexity  involved  with  a 
variety  of  domains,  etc.,  and  that  such  complexity  could  not  all 
be  handled  at  once.  Simplifying  assumptions  had  to  be  made  that 
might  compromise  both  Gagne's  and  Halff's  positions. 


Q_'_Nef  1 '  s  AIDA  Features  Evaluations 

One  way  to  make  the  simplifying  assumptions  recommended  by 
Reigeluth  is  to  use  O'Neil's  evaluation  results.  The  complete 
tabulation  of  O'Neil's  evaluation  matrix  is  included  in  Appendix 
D.  An  initial  interpretation  suggests  the  following: 

1.  Column  11  (feasibility  to  complete  AIDA  as  resourced)  should 
not  be  regarded  as  reliable  due  to  insufficient  responses. 
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2. 


If  columns  2  and  3  (value  to  SME  and  Instructional  Designer, 
respectively)  are  regarded  as  a  measure  of  expected  utility 
to  AIDA  users,  then  there  is  some  agreement  (and  some 
disagreement)  about  what  would  be  useful.  Managers  regarded 
most  features  as  useful;  a  cost-effectiveness  tool  was 
ranked  of  medium  value;  automated  ISD  paperwork  was  ranked 
of  highest  value.  Instructional  designers  also  ranked  a 
cost-effectiveness  tool  of  medium  value,  but  they  ranked  an 
explanation  feature  even  lower;  on-line  documentation  was 
ranked  of  highest  value.  Researchers  ranked  a  cost- 
effectiveness  tool  relatively  low  and  ranked  intelligent 
help  relatively  high. 

3.  The  use  of  templates  was  generally  ranked  of  high  value  to 
the  SME  and  of  low  risk.  An  ITS  for  ISD  and  an  intelligent 
job  aid  were  generally  ranked  of  high  risk. 


Merrill's  Transaction  Shell  Demonstrations 

At  the  February  meeting,  Merrill  had  been  asked  if  he  would  loan 
some  of  his  transaction  shells  to  AFHRL  for  an  evaluation  of 
their  usefulness  in  Air  Force  settings.  Merrill  agreed  in 
principle  and  further  agreed  to  present  a  demonstration  at  the 
April  AIDA  meeting. 

Merrill  prefaced  the  April  demonstration  with  the  need  to  build  a 
library  of  appropriate  transactions  for  a  variety  of  settings. 

He  noted  that  Halff's  claim  that  content  and  interaction  are 
intertwined  goes  against  the  notion  of  second  generation 
instructional  design  (ID-2)  transaction  shells.  Merrill  claimed 
that  there  are  a  limited  number  of  interactions  and  they  are 
fairly  generalizable. 

Merrill's  transaction  shells  grew  out  of  a  CAI  evaluation  study 
which  revealed  that  approximately  600  hours  of  development  time 
were  being  used  to  develop  an  hour  of  very  mediocre  courseware. 
Thv.  transaction  shells  are  written  m  Tujloo  Pascal  2.0.  Merrill 
admits  that  the  demonstration  mode  needs  to  be  made  more 
interactive,  that  constructed  responses  in  practice  sessions  and 
tests  need  to  be  accommodated,  that  an  authoring  capability  for 
some  of  the  shells  needs  to  be  developed,  and  the  graphics 
capability  (currently  Dr.  Halo  graphics  compatible)  needs  to  be 
integrated  into  the  system. 

Merrill  noted  that  it  was  possible  to  control  the  interaction 
parameters,  but  as  yet  there  was  no  guidance  to  advise  users  how 
and  why  parameters  should  be  changed.  It  was  also  possible  to 
limit  or  allow  learner  control,  but  again  there  was  no  guidance 
provided  to  advise  why  and  when  learner  control  was  desirable. 

The  shells  did  comply  with  some  well-established  principles  from 
cognitive  science.  For  example,  it  is  impossible  to  make  a  list 
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or  menu  with  more  than  seven  items  using  Merrill's  nomenclature 
shell,  an  application  of  Miller's  principle  "seven  plus  or  minus 
two"  as  a  production  rule. 

In  addition  to  the  nomenclature  shell,  which  had  a  simple 
authoring  interface,  Merrill  also  demonstrated  a  checklist 
procedure  transaction  shell  and  a  decision-making  transaction 
shell.  The  checklist  procedure  allows  the  user  to  identify  a 
term  and  function  for  a  location  on  a  diagram.  This  simple  shell 
proved  to  be  quite  powerful  by  itself.  Merrill  claimed  that  it 
can  be  easily  modified  to  accommodate  animation.  The  decision¬ 
making  transaction  shell  allowed  the  user  to  create  a 
classification  hierarchy  organized  by  attribute.  The  database 
can  be  searched  by  attribute  and  items  can  be  compared  and 
contrasted. 

The  general  feeling  of  the  group  was  that  Merrill's  shells  show 
great  promise  and  potential,  but  important  questions  remained. 
Merrill  himself  contributed  six  such  questions: 

1.  How  can  transactions  be  designed  to  incorporate  the  kinds  of 
knowledge  they  require  in  order  to  be  executed? 

2.  How  should  course  materials  be  grouped  and  sequenced,  given 
a  particular  set  of  constraints  and  a  content  area? 

3.  What  are  the  different  kinds  of  interaction  modes  and  when 
is  it  appropriate  to  use  each? 

4.  A  display  of  parameter  information  is  needed,  along  with  a 
reconfiguration  capability;  how  can  these  be  provided? 

5.  How  can  transactions  learn  when  and  how  to  call  other 
transactions? 

Merrill  saw  two  big  advantages  in  using  transaction  shells:  1) 
They  contribute  to  fast  and  productive  courseware  authoring,  and 
2)  They  allow  low  level  strategies  to  be  automated  (i.e., 
courseware  developers  need  not  worry  about  paired  associate 
learning) .  AFHRL  will  conduct  a  series  of  studies  to  determine 
the  usability,  generalizability ,  executability ,  and  the 
productivity  of  using  such  tools  in  the  instructional  design 
process . 


Responses  to  the  Critiques 

Tennyson  reviewed  the  revisions  he  had  made  to  his  cycle  2  paper. 
In  the  revision,  Tennyson  has  overtly  dropped  the  notion  of  an 
intelligent  tutoring  system  in  favor  of  an  updated  ISD  process 
with  an  intelligent  interface.  The  revised  notional  structure 
contains  an  ISD  Expert  supported  by  3  functional  components: 
Situational  Evaluation/Diagnosis,  Recommendations/Prescriptions, 
and  a  Mini-expert  to  Guide  Users. 
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The  intelligent  interface  contains  the  ISD  Expert,  an  ISD  Model 
Knowledge  Base,  the  Instructor's  Model  of  ISD,  and  a  Content 
Knowledge  Base  containing  previously  developed  lesson  materials. 

Tennyson  still  envisions  a  system  usable  by  both  novices  and 
experts.  Complete  novices  will  be  "coached"  through  the  system 
by  the  ISD  Expert,  which  will  make  most  of  the  decisions.  More 
advanced  users  will  be  "advised"  through  the  system  with  users 
able  to  preempt  system  defaults  and  decisions. 

Tennyson  also  dropped  the  requirement  for  a  mainframe  and 
restated  that  requirement  in  the  form  of  a  need  for  an  open 
architecture  and  a  centralized,  large  capacity  database  with 
remote  access  via  personal  computers  or  workstations.  He 
estimated  that  it  would  take  about  5  years  and  $10  million  to 
construct  such  a  system.  As  a  consequence,  he  agreed  with 
Reigeluth  and  Halff  that  it  would  be  necessary  to  establish 
simplifying  conditions  in  the  beginning  so  that  within  1  year  a 
minimal  system  prototype  could  be  built.  In  subsequent  years, 
that  initial  system  would  need  to  be  field  tested  and  elaborated. 
Tennyson  stressed  that  his  overall  aim  was  to  build  a  total  ISD 
environment  and  deliver  yearly  growth  products  along  the  way  to 
that  goal. 

In  order  to  make  decisions  about  epitomizing  and  rapid 
prototyping,  Merrill  offered  eight  activities  in  a  possible 
development  plan: 

1.  Choose  the  environment,  users,  and  content. 

2.  Identify  a  number  of  transaction  shells  to  handle  most  of 
the  instruction. 

3.  Identify  initial  parameters  for  those  shells. 

4.  Build  the  initial  shells. 

5.  Test,  tryout,  and  modify  as  necessary  —  loop  #1. 

6.  Add  intelligence  and  mini-experts  —  loop  #2. 

7.  Add  management  system  —  loop  #3. 

8.  Add  integrating  shells  —  loop  #4. 

Merrill  then  warned  of  several  hazards  with  such  a  plan.  First, 
avoid  the  temptation  to  produce  specifications  all  the  way  down 
to  level  #8.  Second  avoid  the  temptation  to  make  specifications 
for  multiple  iterations  at  the  looping  levels.  Third,  make  sure 
that  there  is  a  firm  commitment  to  go  through  level  5  and  one 
loop  in  the  first  attempt. 
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Reigeluth  asked  about  expanding  the  number  of  shells.  Merrill 
replied  that  iterations  through  the  loops  provided  a  mechanism  to 
accomplish  this.  Reigeluth  indicated  that  an  alternative 
elaboration  strategy  would  be  to  begin  the  second  loop  after  a 
single  trip  through  Merrill's  first  loop. 

O'Neil  stressed  the  need  to  build  an  evaluation  system  into  the 
development  plan  and  into  the  system.  If  the  system  collected 
data  to  be  used  for  evaluation,  then  users  could  send  in 
those  data  for  possible  use  in  refining  AIDA. 

Poison's  response  stressed  a  need  to  prioritize  functions  and 
features  proposed  for  AIDA.  She  said  that  we  should  not  view 
AIDA  as  a  complete  authoring  system.  We  don't  need  CMI  in  the 
first-cut,  nor  do  we  need  to  worry  about  automating  paperwork. 

In  fact,  part  of  the  specification  of  the  Advanced  Training  System 
(ATS)  is  to  automate  the  ISD  documentation.  She  agreed  with 
Reigeluth 's  notion  of  epitomizing.  She  saw  a  need  to  have  a 
system  capable  of  handling  a  problem  complex  enough  to  show  that 
AIDA  was  doing  some  good,  yet  the  system  should  be  simple  enough 
to  remain  manageable.  She  agreed  with  O'Neil's  need  to 
prioritize,  but  disagreed  with  him  about  including  metacognitive 
skills  in  a  first-cut  system. 

Poison  supported  O'Neil's  concept  of  commissioning  papers  on  very 
specific  issues  and  problems  in  the  next  phase  effort.  She  said 
that  it  would  not  be  necessary  to  update  Friedman's  paper  unless 
or  until  a  graphics  mini-expert  were  to  be  included  in  AIDA;  she 
did  not  think  the  first-cut  system  should  include  a  graphics 
mini-expert  as  that  would  not  be  an  easily  manageable  task. 

The  checklist  procedure  shell  lacks  a  finished  user  interface. 

Halff's  response  to  the  critiques  by  O'Neil  and  Reigeluth  was 
that  others  had  already  made  his  points,  but  he  indicated  three 
concerns:  1)  Would  AIDA (n)  be  a  smaller  version  of  AIDA(n+i)? 

2)  Would  AIDA(n)  collect  data  to  help  with  AIDA(n+l)?  and  3)  How 
would  users  be  kept  in  the  loop  throughout  the  AIDA  development? 
Halff  liked  Merrill's  cyclic  approach  to  the  development  and 
O'Neil's  idea  to  collect  data  for  the  next  version. 

Muraida  pointed  out  that  one  way  of  keeping  users  in  the  loop  was 
to  insure  that  AIDA  communicated  with  them  in  their  everyday 
language  —  don't  allow  AIDA  to  intimidate  or  alienate  them. 

Merrill  indicated  that  his  tryout  loops  involved  users.  He  took 
the  occasion  to  add  a  fifth  loop  to  his  cyclic  development 
involving  automated  learning  systems.  Such  systems  might  be 
based  on  artificial  neural  networks  and  are  clearly  futuristic  in 
an  instructional  design  setting;  however,  they  are  not 
unimaginable  or  impossible  (see  section  1.1). 
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Halff  again  stressed  that  the  context  in  which  AIDA  is  used  would 
be  the  major  factor  in  determining  what  it  should  be  like. 

Seidel  said  that  academics  did  not  have  an  exclusive  hold  on 
instructional  knowledge;  rather,  there  was  a  great  deal  of  such 
knowledge  in  the  intuitions  and  practices  of  users.  As  a 
consequence,  users  needed  to  be  kept  in  the  AIDA  development  loop 
in  order  to  insure  that  their  knowledge  was  imparted  to  AIDA. 

Tennyson  and  Mongillo  pointed  out  that  aircraft  designers  did  not 
consult  pilots  in  designing  planes  until  the  design  was  fairly 
well  along.  Seidel  countered  that  the  instructional  design 
domain  was  not  so  well  formed  as  aerodynamics.  Halff  added  that 
Merrill's  strategy  was  designed  to  wash  out  useless  ideas  in  the 
iterations  through  the  loops. 

Ellis  and  Merrill  claimed  that  AIDA  could  act  as  a  cognitive 
extender.  Halff  maintained  that,  for  an  explanation,  users  want 
examples  and  lessons  learned,  not  academic  references  to  articles 
in  journals.  Merrill  described  several  levels  of  explanation 
that  could  be  included.  For  example,  one  level  might  be  a  trace 
of  the  rules  used  in  reaching  a  particular  decision;  a  second 
level  might  be  informal  data  in  the  user's  terms.  Merrill  did 
not  see  an  immediate  use  for  O'Neil's  citations  of  formal  studies 
and  experiments. 

Poison  wondered  how  users  would  react  to  a  system  that  helped  a 
lot  with  naming  but  not  at  all  with  using  rules.  Merrill  and 
Poison  agreed  that  one  hazard  of  making  a  nomenclature 
transaction  available  would  be  that  its  ease  of  use  and  power 
might  tempt  users  into  using  it  in  inappropriate  situations.  In 
Dallman's  terminology,  Merrill's  Wrong-Objective-Syndrome  could 
become  Merrill's  Wrong-Transaction-Syndrome. 

Gagne  said  he  was  delighted  at  the  amount  of  agreement  among  the 
consultants.  He  saw  the  group  agreeing  that  we  start  slowly  and 
simply,  collect  and  interpret  data,  and  involve  users  along  the 
way. 


ATC  Views  of  ISP 

Lt.  Col.  Clemons  (ATC/XPCRI)  reviewed  Air  Force  manuals  and 
regulations  pertaining  to  instructional  development.  AFM  50-2, 
which  contains  the  worker  level  description  of  the  five  ISD 
steps,  is  being  revised.  The  new  AFM  50-2  will  incorporate  much 
of  AFR  50-8,  which  mandated  the  ISD  model.  The  applications 
chapters  are  being  removed  from  AFM  50-2,  and  a  new  series  AFP 
50-xx  is  being  started  to  provide  information  pamphlets  on  a 
variety  of  topics.  Clemons  pointed  out  that  the  revision  of  the 
ISD  publications  did  not  imply  that  the  Air  Force  was  revising 
ISD,  however.  In  fact,  Clemons  asked  how  the  group  thought  ISD 
should  be  revised. 
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Tennyson's  cycle  1  paper  is  on  just  that  topic,  and  it  has  been 
provided  to  Clemons  as  a  candidate  for  the  AFP  50-xx  series. 

Gagne  thought  that  'performance'  needed  to  be  well  defined  in  the 
manual.  Merrill  thought  that  enterprises  should  replace 
objectives  in  planning  instruction.  Several  individuals  have 
already  expressed  the  belief  that  any  ISD  material  should  be 
incorporate  to  AIDA  in  a  way  that  is  largely  transparent  to 
users . 

Clemons  also  recommended  targeting  a  second  audience  in  addition 
to  avionics  maintenance  training.  He  indicated  that  targeting 
the  operations  area  would  have  significant  political  Impact  in 
getting  support  for  AIDA.  Therefore,  he  suggested  targeting 
navigation  training  or  undergraduate  pilot  training.  Newcomb 
indicated  a  willingness  to  pursue  this,  but  it  would  have  to  be 
coordinated  with  AFHRL/OT. 

Mongillo  liked  Merrill's  focus  on  the  future.  Rosamond  again 
stressed  the  need  to  keep  users  involved  and  to  consider  novice 
users . 


Seidel's  Presentation  of  Related  ARI  Research 

Dr.  Robert  Seidel  presented  a  review  of  several  projects  in  the 
area  of  automated  training  development  underway  at  ARI.  ARI  has 
published  a  document  on  authoring  system  requirements  and 
guidelines.  ARI  partially  funded  an  earlier  version  of  Merrill's 
ID  Expert.  ARI  and  AFHRL  are  involved  in  the  development  of  the 
Automated  Knowledge  Acquisition  Tool  (AKAT) .  ARI  is  developing 
an  Automated  Systems  Approach  to  Training  (ASAT)  that  will  be 
a  kind  of  automated  ISD  system.  ARI  will  publish  a  cognitive 
model  of  an  expert  author  in  FY  91. 

These  and  other  projects  were  of  obvious  mutual  interest.  All 
agreed  that  ARI,  NPRDC ,  and  AFHRL  needed  to  continue  to  work  in 
close  cooperation  and  to  share  results. 


Consolidation  of  Results 

The  remainder  of  the  afternoon  involved  an  attempt  to  consolidate 
results  and  develop  a  list  of  items  about  which  there  was 
consensus.  Brown-Beasley  claimed  that  we  had  resol”ed  these 
issues : 


1.  The  focus  is  on  a  near  term  AIDA. 

2.  The  next  year's  goal  is  to  implement  an  experimental  vehicle 
which  epitomizes  AIDA. 

3 .  Test  the  system  in  the  field  with  users  involved  in  the 
development . 
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4.  Some  kind  of  knowledge  acquisition  software  would  be  needed 
to  gather  information  about  students,  tasks,  and  the 
environment. 

5.  A  rule  base  for  instruction  is  required. 

6.  An  inference  engine  would  be  required  (e.g.,  CLIPS). 

7.  A  database  with  parameters  to  drive  Merrill-like 
transactions  would  be  required. 

8.  Merrill's  looping  development  approach  should  be  used, 

Halff  thought  there  were  still  unresolved  issues  concerning  what 
AIDA  should  do.  He  did  think  however,  that  we  had  agreed  to 
focus  on  CBT  development  and  design  and  that  we  had  identified 
the  Technical  Training  Center  CBT  developers  as  a  target  user 
audience.  It  was  noted  that  Reigeluth  wanted  the  first-cut  AIDA 
system  to  address  the  analysis  phase  of  ISD  and  that  Merrill 
wanted  the  first-cut  system  to  automate  the  production  of  usable 
instructional  materials. 

Ellis  suggested  that  the  first-cut  AIDA  system  need  only 
incorporate  nomenclature  and  checklist  procedure  transactions. 
Dallman  agreed  that  these  two  transaction  types  mapped  nicely 
onto  the  avionics  maintenance  area. 

Gagne  liked  the  focus  on  maintenance  training,  but  wondered  about 
the  user  interface  and  how  information  about  students,  tasks,  and 
the  environment  would  be  collected.  There  was  considerable 
discussion  about  what  kind  of  explanations  a  first-cut  AIDA 
should  include,  if  any.  This  led  to  a  list  of  issues  that  had 
uoviously  not  been  resolved: 

1.  What  would  be  the  entry  into  the  system? 

2.  What  computer  platform  would  be  used? 

3.  Would  AIDA  provide  tutoring/training? 

4.  Would  AIDA  allow  user  overrides? 

Merrill  distinguished  two  kinds  of  rules  currently  in  his 
transaction  shells:  1)  those  which  were  parametrized,  and  2) 
those  which  were  not  parametrized.  Parametrized  rules  could 
easily  be  overridden  by  users,  if/when  that  was  deemed 
appropriate.  Merrill  then  added  a  few  items  to  the  consensus 
list : 

1.  Rapid  prototyping  ( AFHRL  is  calling  this  incremental 
development) . 

2.  Built-in  data  collection  on  both  instructors  and  students. 
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3.  No  content  semantics  in  the  first-cut  AIDA. 

4.  Use  of  Merrill-like  transactions. 

The  long  term  AIDA  can  then  address  such  items  as  an  intelligent 
advisor,  integrated  instructional  strategies,  an  instructional 
model,  etc. 

There  was  a  great  deal  said  about  satisfying  the  users'  needs  in 
the  near  term  and  delivering  something  useful  to  the  field  as 
soon  as  possible.  There  was  also  agreement  that  we  needed  to 
prioritize  functions  and  features  to  add  as  resources  allowed.  A 
built-in  form  of  data  collection  was  viewed  as  an  excellent  way 
of  collecting  valuable  information  for  evaluation  purposes  during 
the  development  of  AIDA. 

All  seven  academic  consultants  have  agreed  to  participate  in  the 
Phase  II  AIDA  effort,  (Task  13)  Poison  has  indicated  an 
interest  in  doing  a  research  paper  on  issues  related  to  building 
a  transaction  shell  for  teaching  procedural  knowledge.  O'Neil 
indicated  an  interest  in  assisting  in  the  area  of  evaluation. 
Gagne  expressed  an  interest  in  providing  additional  rules  for  a 
first-cut  AIDA. 

The  third  day  of  the  meeting  served  as  a  kick-off  meeting  for  the 
Phase  II  AIDA  effort  (Task  13) . 
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4.4  ARSA&.QE.  gaMg£MSHS..AH.P  PiS&SRSEftEffE 


Here  is  a  summary  of  the  areas  in  which  there  was  agreement  and 
disagreement  during  the  final  AIDA  meeting.  They  are  restricted 
to  the  first-cut  AIDA. 

Areas  of  Consensus 

1.  Focus  on  CBT  development  and  design. 

2.  Focus  on  maintenance  training  at  a  Technical  Training  Center. 

3.  Target  current  Air  Force  CBT  developers. 

4.  Provide  a  help  facility. 

5.  Provide  some  kind  of  explanation  facility. 

6.  Allow  limited  user  overrides  of  defaults. 

7.  Provide  for  the  collection  of  information  about  students, 
tasks,  and  the  environment. 

8.  Use  the  rapid  incremental  development  approach. 

9.  Provide  built-in  data  collection  on  students  and  users. 

10.  Do  not  attempt  to  provide  content  semantics. 

11.  Build  on  the  concept  of  the  transaction  shell. 

12.  Adopt  situated  development. 

13.  Target  intermediate  users. 

14.  Involve  users  in  the  development  process. 

Areas  of  Disagreement 

1.  The  number  and  type  of  shells  required. 

2.  Whether  or  not  to  include  the  analysis  phase. 

3.  Whether  or  not  to  include  non-maintenance  areas. 

4.  What  the  starting  input  to  AIDA  should  be. 

5.  What  computer  platform/architecture  to  use. 

6.  Whether  to  provide  tutoring  and  embedded  training. 
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4 . 5  KEY  PARAMETERS  AND  ISSUES 


Based  on  the  papers  presented  and  the  discussions  at  the  AIDA 
meetings,  21  key  system  attributes  have  been  identified.  The 
attributes  and  the  possible  choices  for  each  are  listed  below. 
The  possible  choices  are  not  intended  to  be  exhaustive. 

Key  System  Attributes  Possible  Choices 


1.  Targeted  Users 

2.  Type  of  Assistance 

3.  Targeted  Subject  Domain 

4 .  Instructional  Design  Theory 

5.  Starting  Input 

6.  Halff's  3  Levels 

7.  Tennyson's  3  Levels 

8.  Control  of  Instructional 
Design  Process 

9.  Priority  Components 

10.  System  Organization 


novice,  intermediate, 
expert  instructional 
designers,  variety 

advising,  coaching, 
constraining  (shells) , 
automated  design  and 
development,  explanations 

maintenance,  electronics 
maintenance,  avionics 
maintenance,  specific 
avionics  device  maint. , 
general  purpose 

ID-1  or  ID-2 

task  analysis,  learning 
analysis,  enterprise 
analysis,  learning 
capabilities,  flexible 

facilitator,  embodier, 
or  implementor 

shells,  sophisticated 
shells,  integrated  system 

user,  system,  mixed 
and  dynamic 

procedure  transaction 
shells,  graphics  mini¬ 
expert,  task  analysis 

Merrill's  6  blocks, 
Tennyson's  ISD-4  phases, 
Reigeluth's  Structure, 
Merrill/Newcomb  diagram, 
Brown-Beasley  structure 
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.Li .  ISD  Scope 

12.  Use  of  AI 

13.  Media  Support 

14.  User  Interface 

15.  Data  Collection 

16.  Databases 

17.  On-line  Support 

18 .  Learning  Theory 

19.  Instructional  Setting 

20.  Organizing  strategy 

21.  Knowledge  Acquisition 


focus  on  design,  develop¬ 
ment,  delivery,  analysis, 
and/or  evaluation  —  in 
existing  AF  model  or  in 
Tennyson's  updated  model 

none,  intelligent  inter¬ 
faces,  expert  advice, 
intelligent  tutor,  expert 
knowledge  acquisition 

graphics  and  text  only, 
full  range  of  multi-media 

menus,  pull-down  menus  and 
buttons,  command  language 
with  macro  capability 

none,  built-in  data 
collection  on  users 

content  materials, 
instructional  rulesets, 
courseware  examples, 
research  citations 

automated  documentation, 
on-line  help,  cost 
effectiveness  tool, 
formative  eval.  tool, 
computerized  measurement, 
authoring  management  and 
storyboarding  tools,  etc. 

update  or  use  existing 
and  established  theories 

computer-based  and/or 
instructor-based 

transaction  theory, 
component-display  theory, 
elaboration  theory,  etc. 

none,  built-in 
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SECTION  5.  PROBLEM  DEFINITION 


The  purpose  of  this  section  of  the  report  is  to  recommend  the 
narrowing  of  problems  to  be  addressed  by  follow-on  AIDA  tasks. 

The  recommendations  are  meant  to  be  consistent  with  the  AIDA  Long 
Range  Plan  (see  section  1.2)  and  are  divided  into  immediate 
concerns  (Task  13,  BAA,  SBIRs,  and  in-house  efforts)  and  long 
range  concerns  (a  fully  specified  contract)  within  these  two 
areas:  1)  scope  or  problem  space,  and  2)  AIDA  features  and 
functions  keyed  to  the  scope  of  the  problem. 


5.1  SCOPE 

The  long  range  vision  of  AIDA  should  remain  fairly  intact.  At 
the  highest  level  of  abstraction,  AIDA  is  an  automated  and 
integrated  collection  of  tools  to  assist  in  the  design, 
development,  and  delivery  of  instruction  (see  section  1.1,  or 
Spector's  original  concept  paper,  "Preliminary  Design 
Considerations  for  an  Advanced  Instructional  Design  Advisor," 
AFHRL/IDC ,  September,  1988).  The  goal  of  AIDA  is  to  provide 
intelligent  and  automated  assistance  throughout  all  phases  of 
instructional  development. 

Of  particular  interest  to  the  AIDA  project  is  computer-based 
instruction.  There  are  a  number  of  reasons  for  this  narrowing  of 
the  scope.  First,  not  much  is  known  about  how  to  optimize 
instruction  to  be  developed  for  new  interactive  technologies. 
Second,  computer-based  instruction  has  great  potential  and 
promise,  if  it  is  designed,  developed,  and  delivered  in  a 
theoretically  based,  empirically  justified,  and  principled 
fashion.  Much  more  is  known  about  how  to  make  instructor-based 
instruction  effective.  What  is  most  needed,  therefore,  are 
effective  instructional  strategies  and  principles  appropriate  to 
new  technologies.  Long  range  plans  should  include  the  widest 
possible  conception  of  computer-based  instruction,  including  all 
existing  and  planned  computer  related  technologies.  Immediate 
plans  should  focus  on  proving  that  automated  tools  can  improve 
the  process  of  designing,  developing,  and  delivering  instruction 
on  existing  microcomputers. 

Another  way  the  scope  should  be  narrowed  is  in  terms  of  subject 
matter  domains.  While  strategies  that  become  validated  as  AIDA 
is  developed  may  prove  to  be  effective  across  a  number  of  subject 
areas,  to  insure  that  some  progress  is  made  that  is  of  value  in 
Air  Force  settings,  the  project  should  focus  on  the  area  of 
maintenance  training.  The  long  range  goal  of  AIDA  should  be  to 
provide  intelligent  and  automated  tools  to  assist  in  the 
development  of  maintenance  training.  Immediate  goals  should  be 
more  focused  on  a  family  of  related  areas,  perhaps  on  electronics 
maintenance  training. 
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A  third  way  that  tha  scope  can  be  narrowed  is  with  regard  to 
users.  The  long  range  goal  is  to  provide  assistance  throughout 
the  ISD  process  to  a  variety  of  users  (instructional  designers, 
developers,  instructors,  students)  at  a  variety  of  levels 
(novice,  intermediate,  expert) .  The  tool  set  would  be 
intelligent  and  adjust  to  the  user  in  appropriate  ways.  Of 
particular  interest  is  the  ability  to  assist  novice  instructional 
designers  develop  effective  computer-based  instruction.  The 
immediate  goal  should  address  intermediate  instructional 
designers  (perhaps  the  typical  TTC  designer)  to  prove  that  useful 
tools  can  be  developed  to  improve  their  productivity  and  also 
improve  student  performance  in  computer-based  settings. 


5.2  KEY  FEATURES 

Section  4  listed  21  AIDA  attributes  gleaned  from  the  various 
papers  and  presentations  by  the  consultants.  Given  the  long 
range  plans  (section  1.2)  and  the  narrowing  of  scope  just 
indicated  (section  5.1),  the  following  table  is  meant  to 
represent  a  coherent  and  rational  selection  of  choices  for 
immediate  and  long  range  planning  purposes: 


Key  System  Attributes 

Immediate 

Long-Term 

1. 

Targeted  Users 

intermediate 

novices,  et  al. 

2  . 

Type  Assistance 

templates,  shells, 
and  explanations 

advising 

3. 

Subject  Domain 

avionics 

maintenance 

maintenance, 
general  purpose 

4. 

ID  Theory 

ID-2 

ID-2 

5. 

Starting  Input 

learning 

capabilities 

flexible  and 
dynamic 

6. 

Half f ' s  3  Levels 

facilitator 
(using  ID-2) 

implementor 

7. 

Tennyson’s  3  Levels 

shells  and  some 

sophisticated 

shells 

integrated 

system 

8. 

Control  of  ID  Process 

primarily  user 

mixed  and 
dynamic 

9. 

Priority  Components 

procedure 

transaction 

shells 

generalizable 
transactions  and 
mini-experts 

10. 

System  Organization 

Merrill 

Merrill/Newcomb 
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11. 

ISD  Scope 

DDD  not  bound 
by  AF  model 

Tennyson  model 
all  phases 

12. 

Use  of  AI 

intelligent 
interface  to 
transaction 
shells 

expert  system 
for  advising, 
intelligent 
knowledge  acq. 

13  . 

Media  Support 

text,  graphics, 
audio,  video 

full-range  of 
support  for 
interactive 
media 

14. 

User  Interface 

pull-down  menus 
with  buttons 

added  command 
lang.,  macros, 
adaptive 

15. 

Data  Collection 

built-in 

built-in 

16. 

Databases 

instructional 
rulesets,  and 
examples 

courseware 
and  content 
expanded 

17. 

On-line  Support 

help,  limited 
authoring  mgt. 
and  storyboard 

expanded  doc . , 
authoring,  and 
cost  mgt. 

18. 

Learning  Theory 

existing  theories 

update  as 
necessary 

19. 

Instructional  Setting 

computer-based 

computer- 

based 

20. 

Organizing  Strategy 

transactions 

integrated 

transactions 

21. 

Knowledge  Acquisition 

guided 

guided 
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SECTION  6.  PROPOSED  SOLUTION  AND  RECOMMENDATIONS 


The  purpose  of  this  section  is  to  provide  preliminary  system 
specifications  for  the  immediate  conception  of  AIDA  —  as  a 
research  tool  and  technology  to  use  in  accomplishing  the  goals  of 
the  near-term  follow-on  tasks  (Task  13,  BAA,  SBIRs,  in-house 
efforts) .  The  first  part  describes  the  concept  in  terms  of  a 
functional  architecture;  the  second  part  provides  functional 
characteristics  and  preliminary  specifications  for  the  near-term 
AIDA;  the  third  part  recommends  a  development  plan  of  action. 


6.1  AIDA'S  FUNCTIONAL  ARCHITECTURE 

The  organization  of  AIDA  proposed  by  Merrill  would  be  capable  of 
providing  the  required  near-term  research  results  (the 
effectiveness  of  various  ID-2  instructional  design  strategies, 
principles,  and  prescriptions) . 

Reigeluth's  organizational  scheme  was  judged  appropriate  for  a 
commercial  instructional  design  setting  involving  a  team  of 
design  and  development  experts.  Tennyson's  organizational  scheme 
based  on  a  cognitively  updated  ISD  model  (ISD-4)  incorporated  a 
coaching  module.  This  would  probably  be  a  worthwhile  enhancement 
to  a  system,  but  does  not  appear  possible  in  the  near-term  due  to 
the  inability  to  model  in  sufficient  detail  instructional  design 
expertise.  However,  many  of  Tennyson's  authoring  activities 
identified  in  his  Appendix  B  can  serve  as  high  level  functional 
characteristics  for  a  near-term  AIDA  (see  especially  B-4  and  B-5) . 

The  modified  Newcomb/Merrill  "virus  diagram"  is  a  reasonable 
organization  scheme  for  the  long-term  view  of  AIDA  (see  Figure 
6.1).  Tennyson's  ISD-4  model  can  also  provide  some  worthwhile 
input  to  a  long  range  model;  Tennyson's  support  of  mini-experts 
adds  credence  to  this  model. 

The  organizational  scheme  best  suited  to  the  near-term  goal 
(building  advanced  transaction  shells  pertinent  to  CBT  in  an  area 
such  as  electronics  maintenance  training)  is  that  provided  by 
Merrill.  However,  Gagne's  advice  to  begin  with  the  results  of  a 
learning  analysis  (learning  capabilities)  allows  the  AIDA  project 
to  focus  on  instructional  strategies  and  transactions,  especially 
as  they  can  be  optimized  for  CBT  in  the  domain  of  electronics 
maintenance  training.  Because  these  strategies  and  transactions 
will  need  to  be  assessed  with  regard  to  their  effectiveness,  a 
delivery  component  will  also  be  needed.  In  short,  the  near-term 
AIDA  organization  concept  can  be  depicted  as  Merrill's  diagram 
modified  to  include  an  evaluation  component  (see  Figure  6.2). 
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AIDA  FUNCTIONAL  CONCEPT 


Four 

Editing 

Support 

Modules 


NOTE: 


Boxes  and  arrows  are  Merrill’s. 
Analyses  are  inputs  to  EXEC. 
Strategies  &  objectives  are  outputs. 
Taxonomies  belong  in  the  EXEC. 


Figure  6.1 
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AIDA  CONCEPT 
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The  concept  of  operations  for  the  near -term  tool  is  that  an 
instructional  designer  with  expertise  in  the  subject  area  but 
limited  knowledge  of  instructional  design  will  input  to  AIDA 
particular  learning  capabilities.  AIDA  will  be  designed  with 
basic  information  about  the  students,  the  environment,  and  the 
task  already  in  the  system.  The  instructional  designer  will  be 
given  the  learning  capabilities  to  enter.  AIDA  will  then  select 
and  configure  transaction  shells  appropriate  to  the  specified 
capabilities.  The  instructional  designer  will  then  be  prompted 
to  enter  any  needed  content  knowledge  to  complete  a  frame 
appropriate  to  the  particular  knowledge  type. 

Because  we  are  assuming  a  fixed  environment  (small  class, 
computer-based  instruction,  located  at  a  TTC)  and  students  who 
are  readers  and  reasonably  motivated,  that  information  can  be 
hard-coded  into  the  AIDA  EXECUTIVE  for  the  time  being.  Because 
we  are  focusing  on  teaching  procedures  for  avionics  maintenance 
training,  we  can  customize  an  enterprise  analysis  pertinent  to 
that  domain  and  also  customize  an  elaborated  frame  network  shell 
pertinent  to  electronics  maintenance  training  procedures. 

Because  these  are  shells  (and  can  also  serve  as  variables  in  AIDA 
experimentation),  they  are  depicted  in  Figure  6.2  as  adjuncts  to 
the  AIDA  EXECUTIVE. 

A  representation  of  this  concept  is  provided  in  Reigeluth's 
model.  Reigeluth's  model  is  an  elaboration  of  Merrill's 
organizational  scheme  in  terms  of  specific  conditions,  methods, 
and  results  involved  in  nine  AIDA  functions. 


6.2  AIDA  SYSTEM  FUNCTIONS  AND  CHARACTERISTICS 


The  near-term  AIDA  experimental  system  proposed  herein  will  be  a 
computer-based  instructional  design,  development,  and  delivery 
(ODD)  assistant  for  training  development  personnel  at  a  TTC  who 
have  some  instructional  design  knowledge  and  experience  with 
courseware  development.  This  system  is  described  below  in  terms 
of  functions  supported,  the  expected  hardware  and  software 
environment,  and  design  details. 

The  near-term  AIDA  will  not  initially  £ .ovide  full  support  for 
all  phases  of  the  instructional  system  development  process.  Some 
phases  will  be  supported  only  in  terms  of  an  epitomizing 
function,  illustrating  how  the  long-term  system  might  function 
and  what  inputs  and  outputs  it  would  provide. 


6*2.1  Summarv_of  AIDA  Characteristics 

Twenty-one  AIDA  characteristics  or  attributes  of  the  near-term 
AIDA,  taken  from  section  5,  are  listed  below.  Some 
characteristics  are  discussed  in  section  6.2.2,  Functional 
Description. 


Key  system  Attributes 

Near-Term 

1. 

Targeted  Users: 

intermediate 

2. 

Type  of  Assistance: 

templates,  shells, 
and  explanations 

3. 

Subject  Domain (s) : 

avionics  maintenance 

4  . 

ID  Theory: 

ID-2 

5. 

Starting  Input: 

learning  capabilities 

6. 

Halff's  3  Levels: 

facilitator  (using  ID-2) 

7. 

Tennyson's  3  Levels: 

shells  and  some 
sophisticated  shells 

8. 

Control  of  ID  Process: 

primarily  user  control 

9. 

Priority  Components: 

procedure  transaction 
shells 

10. 

System  Organization: 

Merrill 

11. 

ISD  Scope: 

DDD  not  bound  by  AF  model 

12. 

Use  of  AI : 

intelligent  interface  to 
transaction  shells 
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13.  Media  Support: 


text,  graphics,  audio, 
video 


14  . 

User  Interface: 

pull-down  menus  with  but 

ions 

15. 

Data  Collection: 

built-in 

16. 

Databases : 

instructional  rulesets, 
examples 

and 

17. 

On-line  Support: 

help,  limited  authoring 
and  storyboard 

mgt . 

18. 

Learr ' ng  Theory : 

existing  theories 

19. 

Instructional  Setting: 

computer-based 

20  . 

Organizing  Strategy: 

transactions 

21. 

Knowledge  Acquisition: 

built-in 

Seven  of  these  requirements  are  described  in  more  detail  in 
paragraphs  6.2.2. 1  through  6. 2. 2. 6.  The  detail  has  been 
abstracted  from  the  consultants'  papers  summarized  in  Section  5. 


6.2.2  Functional  Description 

We  will  use  the  tentative  architecture  (Information,  Content, 
Executive,  Strategies,  Delivery,  and  Evaluation)  described  in 
Section  6.1  to  organize  the  presentation  of  system  functions  in 
this  section.  As  Halff  observed,  AIDA  is  a  description  in 
information  processing  terminology  of  the  design,  development, 
and  delivery  (DDD)  of  instruction.  One  way  to  describe 
information  processing  is  the  functional  flow  diagram.  For 
example,  an  early  functional  flow  diagram  for  instructional 
design  at  the  molar  level  is  shown  in  Figure  6.3  (Bunderson, 

1967) .  A  functional  flow  diagram  specific  to  the  near-term  AIDA 
is  found  in  Reigeluth's  diagram  (Figure  6.4). 

Each  of  the  sections  6.2.2. 1  through  6. 2. 2. 5  is  a  summary  of 
options  for  system  functions  presented  by  individual  consultants. 
The  purpose  of  AIDA  Phase  II  (Task  0013)  is  to  select  from  this 
list  the  functions  and  characteristics  that  will  be  employed  in 
the  near-term  AIDA. 

6. 2.2.1  Information  (Knowledge  Acquisition) 

Information  inputs  required  from  the  user  (i.e.,  the 
instructional  designer)  will  be  elicited  in  easily  comprehended 
ordinary  language.  AIDA  will  not  require  users  to  be  conversant 
with  any  technical  terms  used  below.  The  system  will  guide  the 
developer  through  the  knowledge  acquisition  (KA)  process. 
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Halff  has  distinguished  two  kinds  of  input  to  AIDA: 

1.  Information  about  students,  the  learning  environment,  and 
the  tasks  to  be  mastered. 

2.  Information  about  the  content  material  to  be  mastered. 

Merrill  proposes  collecting  the  following  information  about  the 
audience: 

1.  role  (e.g.,  consumer,  supervisor,  technician,  problem-solver) 

2.  motivation 

3.  familiarity 

4 .  mastery  level 

5.  ability 

Merrill  proposes  collecting  information  about  seven  categories  of 
enterprise  (see  page  37)  . 

1.  denoting 

2 .  executing 

3 .  evaluating 

4.  designing 

5.  manifesting 

6.  interpreting 

7.  discovering 

Merrill  proposes  information  be  collected  about  content  using  the 
elaborated  frame  network  (EFN)  approach: 

1.  frames 

2.  associations  with  other  frames 

3 .  level  in  abstraction  hierarchy 

4.  knowledge  and  skill 

Spector  proposes  information  be  gathered  in  the  following 
categories . 

1.  Enterprise  input  —  a  description  of  the  purposeful  human 
activity  that  is  to  be  trained. 

2.  Task  input  —  a  description  of  specific  tasks  that  comprise 
a  particular  enterprise. 

3.  Learning  objectives  --  descriptions  of  learning  conditions 
and  performance  criteria  appropriate  to  the  training  task. 

4.  Domain  knowledge  base  input  —  the  information  needed  in 
ordev  to  train  the  enterprise. 

5.  Course  organization  —  the  grouping  and  sequencing  of 
information  and  objectives  within  the  training  task. 
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6. 


Transaction  information  —  information  necessary  to  complete 
a  transaction  shell  for  a  particular  learning  objective, 
including  names  of  graphics  files,  desired  levels  of 
interaction,  etc. 

7.  Graphics,  audio,  and  video  input  —  allow  users  to  specify 
file  names  containing  multi-media  support  materials  and 
indicate  where  in  the  course  organization  such  materials  are 
appropriate. 


6. 2. 2. 2  Instructional  Rules  (The  Executive) 

The  rule  base  will  contain  at  least  the  following  principles, 
extracted  from  the  consultants*  concept  papers  as  summarized  in 
section  5. 

o  Gagne's  12  principles  of  instructional  theory  (see  page  13). 

E.g. :  Arouse  student  interest  before  initiating  any 

other  transactions. 

o  Halff's  list  of  cultural  conventions  used  in  instruction 
(see  page  9) . 

o  Kintsch's  10  principles  of  instructional  theory  drawn  from 
research  on  human  cognition  (see  pages  16-17) . 

E.g.:  Center  instruction  on  the  learner's  own  knowledge 

construction  activities. 

Teach  new  knowledge  in  a  meaningful  context. 
Connect  new  knowledge  to  old  knowledge. 

Maximize  learner  involvement. 

Put  learner  in  control  of  the  learning  process. 

Teach  strategies  in  service  of  meaningful  goals. 

o  Merrill's  16  principles  of  pedagogy  and  content  from  ID-2 
(see  pages  13,  20,  21). 

E.g.:  To  aid  later  retrieval,  organize  knowledge  during 

learning . 
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To  aid  later  retrieval,  generate  elaborations  of 
knowledge  during  learning. 

Is  this  an  entity,  activity,  or  process? 

o  Merrill's  12  pedagogical  rules  (see  page  22). 

E.g.:  Learners  follow  the  path  of  least  effort. 

Learners  learn  what  they  do. 

Learning  improves  when  information  is  presented  in 
multiple  ways. 

o  Poison's  7  principles  of  cognitive  psychology  (see  page  25). 

E.g.:  First,  analyze  the  knowledge  structures  of  the 

expert. 

Transform  the  naive  mental  models  of  the  learner 
into  increasingly  more  complex  models  adequate  for 
solving  larger  sets  of  problems. 

To  represent  declarative  knowledge,  use 
propositions,  semantic  nets,  scripts,  schemata, 
frames,  or  plans. 

To  represent  procedural  knowledge,  use  a 
production  system  containing  production  rules  (see 
Newell  and  Simon,  1972) . 

Use  control  processes,  including  metacognitive 
knowledge,  to  guide  the  learner's  processing  of 
knowledge . 

o  Tennyson's  6  cognitive  principles  (see  page  28). 

E.g.:  Use  a  hierarchy  to  relate  a  learning  problem  to 

the  total  learning  context. 

Trace  each  instructional  activity  back  to  needs 
and  goals  of  the  curriculum. 

In  all,  some  100  principles  of  instructional  design  have  been 
extracted  from  the  consultants'  concept  papers.  The  100 
principles  are  not  exhaustive,  but  they  are  a  starting  point. 

The  challenge  in  Phase  II  will  be  to  restate  these  psychological 
aphorisms  as  IF-THEN  rules  useful  in  an  expert  system. 


h  n  n 't  in 


Halff  believes  there  are  two  ways  to  represent  strategies: 

1.  In  a  narrowly  defined  domain:  generative 

2.  Otherwise:  schematic 

Tennyson  listed  five  instructional  strategies  (see  page  27) . 

expository 
practice 
problem-oriented 
complex-dynamic 
self-directed  experiences 

Gagne  suggested  strategies  for  each  of  the  following  three  stages 
of  instruction: 

1.  Setup  period 
Strategy: 

1 .  grabber 

2.  scenario  (enterprise  scenario) 

3 .  reminder 

4.  recall 

2.  Presentation 

Strategy: 

1.  statement 

2 .  example 

3.  label 

4 .  mnemonic 

5.  discourse 

6.  analogy 

7.  elaboration 

8.  human  model 

9.  question 

3 .  Practice 

Strategy; 

1.  practice 

2.  assessment 

3.  telling 

4.  transfer 
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Gagne's  four  steps  in  instructional  strategy  selection: 

1.  Choose  the  media 

2.  Establish  integrative  control 

3.  Classify  the  single  capabilities  to  be  learned 

a.  intellectual  skills 

b.  verbal  information 

c.  cognitive  strategies 
d-  motor  s.  ills 

e.  attitudes 

4 .  Select  and  order  appropriate  instructional  strategies  for 
each  capability 

Halff's  three  representation  schemes: 

1.  If  procedural  knowledge,  then  a  production  system. 

2.  If  declarative  knowledge,  then  a  schema. 

3.  If  causal  knowledge,  then  qualitative  process  theory. 

Merrill  proposes  transactions  as  the  building  blocks  of 
strategies  (see  page  38) .  Transactions  are  real-time,  give-and- 
take  exchanges  between  the  student  and  an  instructional  system. 
The  structure  of  a  transaction  includes: 

1 .  Parameters 

2 .  Interactions 

3 .  Content 

A  transaction  may  be: 

1.  Expository  or  inquisitory 

2.  Lear"  *  or  system  controlled 

3.  "  1  or  experiential 

Transitions  may  be  grouped  as: 

1.  Enterprise  transactions 

2 .  Association  transactions 

3.  Abstraction  transactions 

4 .  Component  transactions 


6. 2. 2. 4  Explanation 

The  near-term  system  will  provide  the  user  (developer)  with  the 
ability  to  request  the  following: 

1.  A  statement  of  the  instructional  design  philosophy 
incorporated  into  AIDA. 
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2 .  A  statement  of  the  roles  of  AIDA  in  the  areas  of  authoring 
management  and  courseware  design,  development,  and  delivery. 

3.  An  epitomizing  description  of  the  targeted  student 
population,  including  skill  levels  and  knowledge. 

4.  An  epitomizing  description  of  the  AIDA  learning  environment. 

5.  Examples  of  enterprise  analyses,  task  analyses,  learning 
objectives,  learning  activities,  and  other  items  required  by 
AIDA  in  order  to  develop  and  deliver  instruction. 

6.  Documentation  of  lesson  and  course  materials,  including 
course  syllabi,  lesson  plans,  study  guides,  and  practice 
problems. 


6. 2. 2. 5  Evaluation 

According  to  Halff,  there  are  three  levels  of  evaluation: 

1.  Quality  control  (Am  I  doing  the  process  right?) 

2.  Formative  evaluation  (Is  it  the  right  process?) 

3.  Summative  evaluation  (Are  our  underlying  assumptions  right?) 

O'Neil  proposes  the  following  steps  in  formative  evaluation. 

1.  Check  design  against  specifications. 

2.  Check  validity  of  instructional  strategies  with  research 
literature. 

3.  Conduct  feasibility  review  with  instructor. 

one-on-one  group  testing 
small  group  testing 

4.  Assess  instructional  effectiveness. 

cognitive 

affective 

5.  Assess  unanticipated  outcomes. 

6.  Conduct  revision. 

O'Neil  lists  the  following  CAI  lesson  evaluation  techniques. 

1.  Quality  review 

-  language  and  grammar  (e.g.,  reading  level) 

surface  features  (e.g.,  uncluttered  displays) 
etc . 

2.  Pilot  testing 

Enlist  about  three  helpers  representative  of  potential 
students . 
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-  Explain  pilot-testing  procedures, 
etc. 

3.  Validation 

Use  the  lesson  in  the  setting  for  which  it  was 
designed. 

Use  the  lesson  with  students  for  which  it  was  designed, 
etc. 


6. 2. 2. 6  Outputs 

The  near-term  system  will  perform  the  following  functions: 

1.  Identify  appropriate  presentation  strategies  for  objectives. 

2.  Identify  appropriate  transactions  to  support  those 
strategies. 

3.  Determine  screen  layouts  and  positioning  whenever  possible. 

4.  Develop  a  lesson  prototype  based  on  specified  objectives, 
content,  and  selected  transaction  shell. 

5.  Provide  the  author  with  a  lesson  test  and  tryout  mode  with 
an  editing  capability. 

6.  Provide  drill  and  practice  and  test  generation  and  grading. 

7.  Collect  data  on  instructor  usage  and  student  performance. 


6.2.3  Design  Details 

Design  details  for  three  crucial  areas  (user  interface, 
productivity  tools,  and  learning  enhancement)  are  provided  below. 


6.2. 3.1  The  User  Interface 

The  primary  user  of  the  near-term  AIDA  will  be  the  courseware 
developer,  although,  in  later  versions,  the  users  may  include  the 
developer's  manager,  other  members  of  the  courseware  development 
team,  and  even  the  student. 

The  critical  factor  in  determining  AIDA's  acceptability  to  the 
developer  is  the  user  interface.  The  user  interface  includes  the 
visual  presentation  of  menus,  choices,  help,  and  information. 

The  user  interface  also  includes  the  semantic  style  and  content 
of  menus,  choices,  help,  and  information. 

The  system  will  begin  with  a  log-on  procedure  which  will 
determine  if  the  user  is  a  developer  or  student.  Figure  6.5 
below  depicts  a  suggested  opening  menu  for  a  developer. 
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-  DEVELOPER’S  MAIN  MENU 
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Figure  6.5 


The  items  listed  on  the  top  line  of  Developer's  Main  Menu  are 
system  choices  that  are  available  throughout  a  development 
session.  The  seven  items  depicted  in  the  middle  of  the  screen 
represent  the  main  steps  through  the  design  and  development 
process.  The  system  will  direct  the  user  through  these  steps  in 
a  sequential  fashion.  The  system  will  indicate  when  a  step  has 
been  completed  (e.g.,  fading  or  a  check  mark).  Completed  steps 
can  be  revisited  for  editing  purposes.  The  bottom  line  will  be 
reserved  for  system  prompts,  cues,  and  general  guidance. 

The  question  mark  button  in  the  lower  left  corner  represents  the 
presence  of  a  dynamic  on-line  help  facility.  The  arrow  button  in 
the  lower  right  corner  represents  a  HOME  button  that  will  take 
the  user  back  to  the  top  level,  offering  an  opportunity  to  save 
any  work  completed.  The  screen  should  also  depict  the  mode 
(developer  or  student)  and  the  current  function  and  location. 

The  diagram  does  not  suggest  how  to  represent  these  items. 

When  an  item  is  activated  by  a  button  click,  pull-down  menus  with 
appropriate  selections  will  appear.  Double  clicking  on  a 
selection  will  provide  the  user  with  the  specified  item.  Icons 
will  be  used  when  appropriate  (e.g.,  in  tool  selection). 

The  student  interface  is  not  depicted.  The  expectation  is  that 
it  will  also  involve  pull-down  menus,  icons,  and  be  button 
driven.  There  will  most  likely  be  fewer  choices  on  the  opening 
menu,  and  there  may  be  fewer  system  choices  across  the  top  menu. 

In  addition  to  providing  an  easily  accessible  visual  interface, 
AIDA  should  provide  the  developer  with  easily  understood  guidance 
through  each  process.  To  accomplish  this,  AIDA  should  provide 
all  guidance  in  ordinary  language  and  avoid  use  of  the  technical 
jargon  of  instructional  design.  In  addition,  AIDA  should  not 
overload  the  user's  short-term  memory.  No  AIDA  menu  should 
contain  more  than  seven  choices  and  nesting  should  be  limited  to 
three  levels  whenever  possible. 

The  developer  will  be  guided  through  the  seven  steps  that 
comprise  the  opening  menu  in  a  straightforward  manner.  Steps 
that  have  been  accomplished  will  be  indicated  and  the  system  will 
automatically  advance  to  the  next  step.  However,  users  will  be 
allowed  to  st^p  back  and  edit  any  previous  step.  Changes  will 
automatically  be  carried  forward  as  appropriate,  and  the  user 
will  be  returned  to  the  next  uncompleted  step  after  an  edit.  If 
all  steps  have  been  accomplished  at  least  once,  the  user  will  be 
returned  to  the  opening  menu. 

It  should  be  noted  that  these  are  recommendations  and  suggestions 
for  the  interface.  The  actual  interfaces  should  be  designed  and 
developed  with  user  involvement  and  consultation. 
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6. 2. 3. 2  Productivity  in  Courseware  Development 


The  critical  factor  in  determining  AIDA's  success  in  improving 
the  productivity  of  courseware  development  is  the  extent  to  which 
course  design,  development,  and  delivery  processes  can  be 
automated.  As  already  indicated,  AIDA  will  automatically  select 
strategies  and  transactions  appropriate  to  those  strategies.  In 
addition,  transaction  shells  will  be  executable  with  a  minimum  of 
input  from  users.  This  automation  can  be  accomplished  with 
extensive  use  of  default  values  for  parameters  associated  with 
transaction  shells. 

Some  parameters  associated  with  transaction  shells  will  be 
alterable  by  users.  As  the  system  evolves,  users  will  be  offered 
more  control  and  advice  about  various  ways  transaction  shells  can 
be  customized  and  optimized  for  particular  purposes.  Users  will 
have  access  to  an  explanation  facility  that  provides,  for 
example,  a  justification  for  beginning  a  procedural  lesson  with  a 
scenario . 

Transaction  shells  in  each  of  Gagne's  three  stages  (setup, 
presentation,  and  practice)  should  be  provided  in  the  initial 
system.  In  addition,  the  initial  system  should  attempt  to 
provide  templates  for  as  many  of  Gagne's  strategies  as  possible. 
For  example,  when  the  user  is  sequencing  a  lesson,  AIDA  should 
provide  the  user  with  a  setup  menu  and  offer  a  grabber  or 
scenario  template  followed  by  a  reminder  or  recall  transaction 
shell . 

In  addition  to  executable  transaction  shells,  AIDA  should  provide 
easy  access  to  multi-media  support.  A  standard  graphical  user 
interface  with  all  parts  of  the  system  accessible  through  that 
interface  will  contribute  to  improved  productivity  in  this  area. 


6. 2. 3. 3  Enhancement  of  Student  Learning 

The  critical  factor  in  determining  AIDA's  contributions  to 
student  comprehension,  retention,  and  transfer  is  the  optimality 
of  instructional  strategies  and  prescriptions  provided  for 
various  training  objectives.  This  initial  AIDA  system  is 
intended  as  an  experimental  system.  Data  will  be  collected  to 
determine  the  optimality  of  the  initial  instructional  theory  rule 
base.  When  evidence  suggests  that  strategies,  transactions,  and 
sequencing  selected  by  AIDA  are  less  than  optimal,  appropriate 
modifications  will  be  made  to  the  rule  base. 
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6-2.4  Expected  Hardware/Software  Environment 


It  is  generally  agreed  that  having  software  products  that  can  be 
transitioned  to  the  field  as  they  are  developed  and  validated  is 
worthwhile.  The  first  executable  AIDA  product  is  scheduled  to  be 
delivered  in  FY  92  with  an  experimental  testbed  system  completed 
in  FY  94. 

The  Air  Force  TTCs  primarily  use  MS-DOS  3.0  Zenith  Z-248 
microcomputers  with  EGA  monitors  for  CBT.  It  is  expected  that 
these  will  be  gradually  upgraded  to  80386  systems  with  higher 
resolution  monitors  and  additional  peripheral  devices. 

As  a  consequence,  Mei  Associates  recommends  that  the  targeted 
computer  environment  for  the  near-term  should  be  80386  based 
systems  with  VGA  monitors.  Efforts  should  be  made  to  develop  a 
subset  AIDA  that  could  run  under  MS-DOS  3.0  on  the  Z-248s  with 
EGA  monitors. 

Due  to  the  expected  use  of  many  multi-media  peripherals  and  the 
need  for  a  standard  graphical  user  interface  (GUI) ,  it  is 
recommended  that  AIDA  be  targeted  for  a  GUI  that  will  be  widely 
available  for  the  targeted  hardware.  Microsoft  Windows  3.0, 
Hewlett-Packard  NewWave,  and  OS/2  Presentation  Manager  are 
possible  GUI  candidates. 

AFHRL  should  conduct  an  evaluation  in  FY  91  to  determine  the 
exact  hardware  and  software  environment  desired  for  the  near-term 
AIDA  system. 
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SECTION  7.  RESEARCH  ISSUES 


The  purpose  of  this  section  is  to  identify  specific  research 
issues  that  pertain  to  AIDA.  Research  issues  are  divided  into 
immediate  and  long-term  concerns. 


7-1  NEAR-TERM  RESEARCH  QUESTIONS 

The  following  research  questions  were  raised  in  cycle  1. 


Research  Issues  Identified  in  Cycle  1 

1.  How  should  AIDA  make  use  of  artificial  intelligence? 

2 .  What  particular  functions  and  principles  should  be  tested  in 
an  initial  prototype? 

3.  What  degree  of  specificity  is  required  to  implement  a  major 
functional  component  of  AIDA  (e.g.,  mapping  learning 
objectives  to  micro-  and  macro-instructional  strategies)? 
What  knowledge  bases  and  components  are  required  to 
accomplish  this? 

4 .  By  what  process  should  new  research  be  incorporated  into 
AIDA's  instructional  design  knowledge  base? 

5.  Can  AIDA  provide  instructional  design  advice  across  a  wide 
variety  of  content  areas?  How? 

6.  To  what  extent  can  expert  instructional  design  advice  be 
automated? 

7.  To  what  extent  can  instruction  be  generated  automatically 
based  on  the  instructional  design  advice? 

8.  How  does  instruction  becomes  progressively  constrained  as 
content  becomes  progressively  specialized. 

9.  How  appropriate  (useful,  acceptable,  etc.)  is  a  top-down, 
hierarchically  dependent  development  process. 

10.  Why  is  traditional  ISD  dead  in  the  DoD? 

11.  Why  didn't  the  Air  Force  adopt  ISS? 

12.  What  success  stories  for  authoring  systems  exist? 

13.  What  are  the  cost  drivers  and  tradeoffs  pertinent  to 
authoring  systems? 
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The  following  research  issues  were  identified  in  cycle  2. 


Research  Issues  Identified  in  Cycle  2 

1.  What  transaction  snails  need  to  be  included? 

2.  What  representational  scheme  for  the  knowledge  base  will  be 
useful  for  instructional  purposes? 

3.  How  can  a  representation  scheme  be  mapped  onto  transactions 
in  ways  thac  guide  the  presentation  of  instruction  to 
students? 

4.  What  are  the  appropriate  interaction  modes  within  a 
transaction? 

5.  what  transactions  can  be  collected  to  build  a  database  of 
examples  and  to  use  in  abstracting  common  characteristics? 

6.  What  rules  can  be  found  in  the  instructional,  computer 
science,  and  cognitive  science  literatures  that  fit  into  our 
model? 

7.  How  do  we  show  that  we  are  improving  learning? 

8.  [How]  Does  knowledge  representation  lead  to  improved 
learning? 

9.  [When]  Does  interactivity  result  in  improved  learning? 

10.  What  features  of  a  CBT  environment  contribute  to  improved 
1  earning? 

11.  What  determines  a  person's  orientation  to  visual  or  verbal 
information?  How  do  we  determine  a  person's  orientation? 

12.  How  can  transactions  be  designed  to  incorporate  the  kinds  of 
knowledge  they  require  in  order  to  be  executed? 

13.  How  should  course  materials  be  grouped  and  sequenced,  given 
a  particular  set  of  constraints  and  a  content  area? 

14.  What  are  the  different  kinds  of  interaction  modes  and  when 
is  it  appropriate  to  use  each? 

15.  A  display  of  parameter  information  i:  needed,  along  with  a 
reconfiguration  capability;  how  can  these  be  provided? 

16.  How  can  transactions  be  designed  so  that  they  i 
communicate  with  other  transactions? 
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7 . 2  LONG-TERM  RESEARCH  QUESTIONS 


Many  of  the  near-terra  research  questions  indicated  above  will 
continue  to  merit  attention  in  the  long  term-  This  section 
presents  additional  long-term  research  issues. 

AIDA's  research  agenda  consists  of  two  major  streams: 

1.  Validation  of  instructional  strategies,  principles,  and 
prescriptions  with  new  interactive  technologies. 

2.  Development  of  simple,  effective  methods  of  presenting 
online  instructional  design  guidance  to  novice 
instructional  designers. 

The  two  streams  are  interdependent  in  that  useful  instructional 
design  guidance  must  address  the  new  interactive  technologies 
available  to  CBI  designers  while  ascertaining  that  the  method 
of  presenting  the  guidance  online  minimizes  cognitive  and 
ergonomic  demands  on  the  novice  user.  The  value  of  AIDA  will 
increase  to  the  extent  that  research  can  coordinate  these  two 
concerns . 

With  respect  to  instructional  strategies,  one  of  the  first 
issues  that  should  be  investigated  is  what  strategies  are 
actually  taught  on  a  wide  basis.  Recent  observational  research 
has  pointed  out  the  need  for  an  inventory  of  instructional 
strategies  actually  being  used  in  public  education  (Pressley,  et 
al,  1990).  It  would  be  extremely  useful  to  find  out  what 
instructional  strategies  are  in  use  in  military  settings,  coupled 
with  data  on  their  effectiveness.  This  would  provide  the  basis 
for  retaining  the  best  of  current  military  instruction  in  the 
AIDA  knowledge  base.  The  issue  of  determining  which  strategies 
(whether  explicitly  taught  or  not)  are  effective  has  been 
complicated  because  of  research  design  flaws  which  fail  to  rule 
out  alternative  causes  for  performance  effects.  One  of  the  most 
important  questions  about  the  success  of  instructional  strategies 
is  the  degree  to  which  they  are  influenced  by  individual 
differences  in  cognitive  processes  such  as  short-term  memory 
capacity,  intelligence,  or  expertise  in  a  content  area.  Coupled 
with  this  question  is  the  related  issue  of  whether  instructional 
strategies  are  differentially  effective  with  culturally  different 
populations.  Quite  apart  from  the  answers  to  the  strategy 
effectiveness  questions  it  remains  for  research  to  address  the 
issues  of  which  procedures  will  most  effectively  teach  the 
strategies  and  which  procedures  will  produce  long-term  strategy 
use.  The  guidance  AIDA  can  provide  will  be  immeasurably  more 
valuable  to  the  novice  designer  if  answers  to  the  above  issues 
can  be  located  in  the  literature  or  derived  from  empirical 
work . 

The  question  of  conveying  online  instructional  design  guidance 
to  the  novice  designer  revolves  around  the  relationship  between 
user  characteristics,  interface  configurations,  and  the  resulting 
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CBI  quality  and  effectiveness.  One  of  the  most  pressing  issues 
is  the  degree  to  which  minimal  user  (novice  designer)  training 
inn  computer  software  use  and  instructional  design  can  compensate 
for: 

1.  Deficiencies  in  those  areas  in  cases  where  the  authoring 
environment  has  no  major  flaws. 

2 .  Flaws  in  the  authoring  environment  in  cases  where  users 
have  experience/ability  deficiencies. 

A  related  question  is  whether  training  the  designer  in  cognitive 
and  metacognitive  strategies  will  improve  the  quality  of  use,  the 
resulting  quality  of  CBI,  and  its  effect  on  student  performance. 
Finally,  it  would  be  useful  to  determine  the  possibility  of 
matching  user  ability  and  experience  profiles  with  particular 
authoring  environment  configurations.  The  purpose  of  this 
effort  would  be  to  minimize  hardware/software  development 
system  and  instructional  costs,  while  optimizing  the  quality  of 
instructional  development  and  student  performance.  Informative 
studies  relevant  to  these  issues  could  make  it  possible  in  the 
foreseeable  future  to  customize  instructional  design  authoring 
environments  for  designers  of  different  backgrounds  and 
abilities,  allowing  them  to  make  optimal  use  of  instructional 
prescriptions  and  technologies. 
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Functional  Characteristics  -  Assignments 

Dr.  Muraida 

12:00 

Lunch  at  the  Officers  Club 

1:30 

Research  Agenda  -  Task  Assignments 

Dr.  Spector 

2:45 

Short  Break 

3:00 

Procedural  Issues  -  AIDA  Follow-ons 

Dr.  Muraida 

3:45 

Short  Break 

4:00 

Observations  of  Military  Advisors 

Dr.  Spector 

4:30 

Concluding  Remarks 

Dr .  Newcomb 

5:00 

Adjourn 
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Third  Concept  Design  Meeting 
February  20-21,  1989 
Air  Fcrce  Human  Resources  Laboratory 
Brooks  AFB,  San  Antonio,  Texas 

Attendees 


Dr. 
Dr . 
Dr. 
Dr. 


Consultants 
Dr.  Robert  M.  Gagne 
Henry  M.  Halff 
M.  David  Merrill 
Martha  C.  Poison 
Robert  Tennyson 
AF  Human  Resources 
Dr.  Scott  Newcomb 
Dr.  Daniel  Muraida 
Dr.  J.  Michael  Spector 
MAJ  Bill  Wimpee 
Ms.  Barbara  Eaton 
Military  Advisors 
Dr.  Jonn  A.  Ellis 
LCOL  Jerry  Barucky 
LCOL  Dan  Meigs 
LCOL  Mike  Bush 
Dr.  Mary  Marlino 
LCOL  Rich  Ranker 
Mr.  Brian  Dallman 
LCOL  Mike  Dickinson 
CAPT  James  Coward 
MAJ  Robert  Mongillo 
MAJ  Karen  Reid 
Dr.  Ok-choon  Park 
Mei  Associates 
Dr.  Albert  E.  Hickey 
Mr.  Richard  Vigue 


Florida  St.  Univ 
Halff  Resources, 
Utah  State  Univ. 
Univ.  of  Colorado 
Univ.  of  Minnesota 


Inc . 


Laboratory 
AFHRL 
AFHRL 
AFHRL 
AFHRL 
AFHRL 


NPRDC 

USAFRS/RSCD 
3302d  TCHTS/CC 
USAFA/DFF 
USAFA/DFT 
ACSC/EPD 

3400  TCHTW/TTGCX 

Advanced  Training  System 

HQ  ATC/XPCRI 

ATC/XPC 

ATC/TTIP 

Army  Res.  Institute 

Project  Manager 
Mei  Associates/SA 


Agenda 

Tuesday,  February  20,  Presentation  of  Cycle  2  Tasks 


8:30 

Welcome  Back 

Dr . 

Newcomb 

8:45 

Goals/Agenda 

Dr. 

Muraida 

9:00 

Presentation  of  Task  Results 

Dr . 

Muraida  (moderator) 

9:00 

Dr.  Robert  M.  Gagne 

10:00 

Break 

10:30 

Dr.  Henry  M.  Halff 

11:30 

Lunch 

(more . . . ) 

I  OH 


1  :  0  0 

Dr.  M.  David  Merrill 

2:00 

Short  Break 

2  :  15 

Dr.  Martha  C.  Poison 

3  :  15 

Short  Break 

3:30 

Dr.  Robert  D.  Tennyson 

4  :  30 

Wrap-up/Observations 

Dr . 

Spector 

5  :  00 

Adjourn  for  the  day 

Dinner  on  your 

*  *  * 

own  or  at 
★  *  *  _  ^ 

Tomatillo' s 

Wednesday, 

Feb.  21:  Intearatincr  Task  Results  for 

the  Final  Report 

8  :  30 

Architectural  Summary 

Dr. 

Spector 

(moderator) 

9:  15 

Break 

10  :  15 

Functional  Characteristics 

Dr. 

Muraida 

(moderator) 

11:00 

Research  Issues 

Dr. 

Hickey 

(moderator) 

11:30 

Lunch 

1:00 

AIDA  Phase  II  (Task  0013)  Requirements 

Dr. 

Muraida 

1:30 

From  Func  Specs  to  S/W 

Specs 

Ms . 

Barbara 

Eaton 

2  :  00 

Short  Break 

2:15 

Tasking  the  Reviewers 

Dr. 

Spector 

3:15 

Short  Break 

3:30 

Wrap-up/Observations 

Dr . 

Hickey 

(moderator) 

4:30 

Adjourn 
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Final  Concept  Review  Meeting:  Phase  I 
April  24-25,  1990 
Kick-Off  Meeting:  Phase  II 
April  26,  1990 

Air  Force  Human  Resources  Laboratory 
3rooks  AF3,  San  Antonio,  Texas 

Attendees 


Consultants 
Dr.  Robert  M.  Gagne 
Dr.  Henry  M.  Halff 
Dr.  M.  David  Merrill 
Dr.  Martha  C.  Poison 
Dr.  Robert  Tennyson 
Dr.  Harold  F.  O'Neil 
Dr.  Charles  Reigeluth 
AF  Human  Resources  Lab' 
Dr.  Scott  Newcomb 
Dr.  Daniel  Muraida 
Dr.  J.  Michael  Spector 
MAJ  Bill  Wimpee 
Military  Advisors 
Dr.  John  A.  Ellis 
LCOL  Jerry  Barucky 
LCOL  Dan  Meigs 
LCOL  Mike  Bush 
Dr.  Mary  Marlino 
LCOL  Rich  Ranker 
Mr.  Brian  Dallman 
LCOL  Mike  Dickinson 
CAPT  James  Coward 
MAJ  Robert  Mongillo 
MAJ  Karen  Reid 
Dr.  Ok-choon  Park 
LCOL  Larry  Clemons 
Dr.  Dee  Andrews 
Mr.  John  LaBarbera 
Mei  Associates 
Dr.  Albert  E.  Hickey 
Mr.  Richard  Vigue 


Florida  St.  Univ. 

Halff  Resources,  Inc. 
Utah  State  Univ. 

Univ.  of  Colorado 
Univ.  of  Minnesota 
Univ.  of  S.  California 
Indiana  University 
oratory 

AFHRL/IDC 

AFHRL/IDC 

AFHRL/IDC 

AFHRL/IDC 

NPRDC 

USAFRS/RSCD 
3302d  TCHTS/CC 
USAFA/DFF 
USAFA/DFT 
a  /Fpn 

3400  TCHTW/TTGCX 
HQ  HSD/YA 
HQ  ATC/XPCRI 
ATC/XPC 
ATC/TTIP 

Army  Research  Institute 

ATC/XPCRI 

AFHRL/OT 

HQ  USAF/DPPE 

Project  Manager 
Mei  Associates/SA 


Mr.  Michael  Brown-Beasley  Mei  Associates/Lex 

Agenda 


Tuesday 


>ril  24 :  Critique  of  Cycle  2  Tasks 


8:30 


Welcome  Back 


Dr.  Newcomb 


8:45 


Goals /Agenda 


Dr.  Muraida 


9:00 


Summary  of  AIDA  To  Date 


Dr.  Spector 


1 
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9:30 

Demos  of  AIDA-related  Systems 

Dr. 

Mur a i da 

(moderator) 

9:30 

Demo  of  IDiOM 

Dr. 

Mariino 

10:30 

Break 

11:00 

Demo  of  QUEST 

Mr. 

Dallman 

12:00 

Lunch  at  the  Officers'  Club 

1 :  30 

Demo  of  SOCRATES 

Dr. 

Specter 

2:00 

1st  Critique  of  Cycle  2 

Dr. 

O'Neil 

3:00 

Break 

3:30 

2nd  Critique  of  Cycle  2 

Dr. 

Reigeluth 

4:30 

Responses  to  Critiques 

Dr. 

Muraida 

(moderator) 

4:30  Dr.  Gagne 

4:45  Dr.  Halff 

5:00  Dr.  Merrill 

5:15  Dr.  Poison 

5:30  Dr.  Tennyson 

5:45 

Adjourn  for  the  day 

_  _  __  ★  —  —  — 

Dinner  on  your  own  or 

on 

the  river 

_  ***  w- 

— 

Wednesday 

.  Aoril  25 :  Conclusions  for  the  Phase  I  Final  Report 

8:30 

Summary  of  the  Critiques 

Dr. 

Spector 

(moderator) 

9:00 

Research  Issues 

Dr. 

Muraida 

(moderator) 

9:30 

Transaction  Shell  Demo 

Dr. 

Merrill 

10:00 

Discussion  of  Transactions 

Dr. 

Merrill 

10:30 

Break 

11:00 

Final  Review  of  AIDA  Specs 

Dr. 

Spector 

(moderator) 

12:00 

Lunch  at  the  Officers'  Club 

1:30 

Final  Review  of  AIDA  Specs 

Dr. 

Spector 

(moderator) 

3:00 

Break 
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3  :  30 

Final  Review  of  AIDA  Specs 

Dr  . 

Spector  (moderator) 

5:00 

Adjourn  for  the  day 

- _ _  *  *  *  —  . 

_ 

Dinner  on  your  own  or 

on 

the  river 

Thursday, 

April  26:  Kick-off  Maetinar  for  AIDA  Phase  II  (Task  0013) 

8:30 

Review  of  the  Agenda 

Dr. 

Muraida 

8:45 

Description  of  AIDA  Phase  II 

Dr. 

Muraida 

9:00 

Overall  Plan  of  AIDA  Phase  II 

Dr. 

Mr. 

Hickey 

Brown-Beaslev 

1 .  General  Procedure 

2.  Knowledge  Acquisition 

3.  Expert  System  Architecture 

4.  Demonstration  of  the  AIDA  Model 

10:00  Break 


10:30 

Phase  II  Milestones 

Dr. 

Muraida 

1.  Cycle  1:  Design  Specs 

2.  Cycle  2:  Design  Specs 

3.  Final  Reports 

for  Knowledge  Bases 

for  Linking  Knowledge  Bases 

12:00 

Lunch  at  the  Officers'  Club 

1:30 

Role  of  the  Consultants  in 
Phase  II 

Dr. 

Hickey 

(moderator) 

2:30 

Break 

3:00 

Addressing  Research  Issues 

Dr. 

Spector 

(moderator) 

4:00 

Developing  a  Test  Plan 

Dr. 

Muraida 

(moderator) 

5:00 

Adjourn 
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APPENDIX  B 


TAKING  ASSIGNMENTS 

Table  of  Contents 


Cycle  1 

Desic 

jn  Tasks 

1 . 

Gagne 

2. 

Halff 

3. 

Merrill 

4  . 

Poison 

5. 

Tennyson 

Review  Tasks 

6  . 

O'Neil 

7. 

Reigeluth 

1.  Uniform  Tasking  Assignment  for  Gagne,  Halff,  Merrill, 
Poison  and  Tennyson 

Review  Tasks 

1.  O'Neil 

2.  Reigeluth 
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PROJECT  AIDA 


SPECIFICATIONS  FOP  CONSULTANT  TASKS 
First  Design  Cycle 
(Revised  10  August  1989) 

AIDA  seeks  to  develop  instructions  and  job  aids  that  will  make 
instructional  design  efficient,  particularly  for  those  who  are 
relatively  inexperienced,  and  whose  knowledge  of  the  theoretical 
basis  for  instructional  design  is  limited. 

AIDA  seeks  to  (a)  define  the  nature  and  scope  of  relevant 
learning  theory,  knowledge  theory,  and  instructional  theory,  and 
(b)  to  bring  about  a  systematic  integration  of  these  three 
domains  as  a  foundation  for  instructional  design. 


_  *  *  *  _ 

Task  1,  Poison:  State  the  ten  or  twelve  major  propositions 
derived  from  cognitive  learning  research  which  would  probably 
be  agreed  to  by  leading  investigators  in  the  field,  and  which  have 
an  essential  relation  to  instructional  design.  Describe  in  terms 
as  simple  as  possible  why  each  principle  is  important  to 
inct-uct  .nc.1  Jasigii,  and  what  effect  knowing  it  may  be  expected 
to  have  on  the  performance  of  the  instructional  designer. 

Task  2,  Poison:  State  the  major  distinguishing  characteristics 
of  the  contents  of  cognition,  including  the  different  kinds 
of  knowledge  and  the  different  forms  each  type  may  take.  Consider 
in  your  description  the  contributions  of  epistemologists  (e.g., 
Michael  Polanyi)  and  cognitive  psychologists  (e.g.,  Anderson,  Simon). 
Restrict  your  answer  to  the  forms  of  cognition  which  carry  clear 
implications  for  the  nature  of  instruction  and  its  design.  As  an 
aim,  describe  what  kind  of  "knowledge  theory"  must  form  a  basis 
for  the  design  of  instruction. 

Task  3 ,  Gagne :  State  the  twenty  (or  so)  principles  of 
instructional  theory  (as  exhibited  for  example  in  Reigeluth' s 
book)  which  can  be  commonly  held  by  all  instructional  theorists, 
and  concerning  which  there  is  minimal  or  zero  disagreement,  except 
perhaps  in  priority  of  importance.  Indicate  common  ideas  and  variant 
terminology;  identify  reasons  for  differences  in  terminology  when 
possible.  Consider  the  relative  importance  of  these  principles 
in  determining  the  nature  of  instruction  and  the  process  of 
instructional  design.  When  appropriate,  indicate  differences  in 
the  depth  of  knowledge  about  learning  required  by  the 
instructional  designer  when  using  each  of  these  principles  in 
designing  instruction. 

Task  4,  Halff:  Provide  a  concept  of  how  AIDA  would  support  the 
design  of  instruction  in  two  broad  instructional  paradigms  to  be 
selected.  And  identify  the  principles  of  learning  and  instruction 
that  apply  to  the  design  of  the  two  AID As  and  to  their  instructional 
products . 


Tajik — Merrill :  Define  Merrill'  3  concept  of  AIDA:  the 

functions  it  should  perform,  including  knowledge  acquisition  (see 
Task  2  above)  and  strategy  analysis  (see  Task  3  above)  .  And 
identify  the  principles  of  knowledge  theory,  learning  theory,  and 
instruction  theory  that  underlie  these  functions. 


6f — Tennyson:  Update  ISD  model  from  advances  in  cognitive 
science,  AI,  and  educational  technology. 


'T^ocn^&r  A/ £>4 

(Lt>&ie&'c-/977  <oaj  3c#&yr/er 
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MEI  ASSOCIATES,  INC. 

1050  WALTHAM  STREET.  LEXINGTON,  MASS  02173 
(617)  862-) 390 


August:  11,  1989 


Dr.  M.  David  Merrill 
Instructional  Technology  Dept. 

Utah  State  University 
Logan,  Utah  84322-3025 

RE:  Project  AIDA 

Dear  Dave : 

Now  that  the  three  kick-off  meetings  have  been  completed, 
we  would  like  to  summarize  your  task  assignment  for  cycle  1. 

As  you  proposed,  your  primary  task  is  to  "define  (your)  concept  of 
AIDA:  the  functions  it  should  perform,  including  knowledge 

acquisition  (see  Task  2)  and  strategy  analysis  (see  Task  3) .  And 
identify  the  principles  of  knowledge  theory,  learning  theory,  and 
instruction  theory  that  underlie  these  functions."  That  is 
Task  5  on  the  attached  list. 

To  help  us  correlate  the  results  of  Tasks  4,  5,  and  6, 
please  make  a  separate  list  of  principles  derived  from  research 
on  cognition  and  cognitive  processes  involved  in  learning  and 
relevant  to  your  primary  task.  (This  is  akin  to  the  subtask 
described  in  the  second  sentence  of  your  own  task  definition.  It 
is  also  an  abbreviated  version  of  Gagne's  Tasks  1  and  2.) 

The  overall  process  of  correlation,  shown  in  the  attached 
schematic,  will  begin  when  you  submit  your  draft  on  October  10 
and  continue  until  we  send  the  integrated  results  to  Harry  O'Neil 
and  Charles  Reiyaiuth  on  October  31. 

Meanwhile,  as  you  suggested,  HRL  will  concurrently  prepare: 

1.  A  needs  statement 

2 .  A  requirements  document 

3.  A  general  specification,  or  vision  of  AIDA 

On  another  subject,  Scott  Newcomb  says  that  the  JAG  at  Brooks 
AFB  has  declared  that  your  work  on  thi3  task  will  not  preclude 
your  bidding  on  any  future  procurements  connected  with  AIDA. 


Finally,  if  is  agreeable  with  you,  the  meeting  in  October 


will  be  moved  up  one  cay  to  Tuesday  and  Wednesday,  October  17, 
18,  to  avoid  a  conflict.  Martha  Poison  is  scheduled  to  chair  a 
panel  at  the  HFS  on  tie  19th. 

We  believe  the  kick-off  meetings  have  been  productive  and 
augur  well  for  the  AJEA  project.  I  look  forward  to  receiving 
your  draft  on  October  10.  If  you  have  any  questions,  please 
call  me  directly  at  617-862-3390,  or  fax  617-862-5053. 


Sincerely  yours. 


Albert  E.  Hickey,  PhD 


Enel .  : 

Specifications  of  Consultant  Tasks  (6) 
Schematic  of  Correlation  Process 
Letters  to  other  consultants  (4) 


MEl  ASSOCIATES.  INC. 


IQ»  WALTHAM  STHEET  LEXINGTON  .  VIAOJI'I  (6in  (USJ  JV*> 


ME  I  ASSOCIATES,  INC. 

1050  waltham  street,  lexington.  mass.  0:1:3 

(61?) 862-3390 


August  11,  1989 


Dr.  Robert  D.  Tennyson 
178  Pillsbury  Drive  S.E. 

University  of  Minnesota 
Minneapolis,  Minnesota  55155 

RE:  Project  AIDA 

Dear  Bob: 

Now  that  the  three  kick-off  meetings  have  been  completed, 
we  would  like  to  summarize  your  task  assignment  for  cycle  1. 

As  you  proposed,  your  primary  task  is  to  "update  the  ISD  model 
from  advances  in  cognitive  science,  AI,  and  educational 
technology . ”  That  is  Task  6  on  the  attached  list. 

To  help  us  correlate  the  results  of  Tasks  4,  5,  and  6, 
please  make  a  separate  list  of  principles  derived  from  research 
on  cognition  and  cognitive  processes  involved  in  learning  and 
relevant  to  your  primary  task.  (This  is  an  abbreviated 
version  of  Gagne's  Tasks  1  and  2.) 

The  overall  process  of  correlation,  shown  in  the  attached 
schematic,  will  begin  when  you  submit  your  draft  on  October  10 
and  continue  until  we  send  the  integrated  results  to  Harry  O'Neil 
and  Charles  Reigeluth  on  October  31. 

Meanwhile,  as  recommended  by  the  consultants,  HRL  will 
concurrently  prepare: 

1.  A  needs  statement 

2.  A  requirements  document 

3.  A  general  specification,  or  vision  of  AIDA 

On  another  subject,  Scott  Newcomb  say3  that  the  JAG  at  Brooks 
AFB  has  declared  that  your  work  on  this  task  will  not  preclude 
your  bidding  on  any  future  procurements  connected  with  AIDA. 

Finally,  if  it  is  agreeable  with  you,  the  meeting  in  October 
will  be  moved  up  one  day  to  Tuesday  and  Wednesday,  October  17, 

18,  to  avoid  a  conflict.  Martha  Poison  is  scheduled  to  chair  a 
panel  at  the  HFS  on  the  19th. 


We  believe  the  kick-off  meetings  have  been  productive  and 
augur  well  for  the  AIDA  project.  I  look  forward  to  receiving 
your  draft  on  October  10.  If  you  have  any  questions,  please  call 
me  directly  at  617-862-3390. 


Sincerely  yours. 


Albert  F .  Hickey,  PhD 


Enel .  : 

Specifications  of  Consultant  Tasks 
Schematic  of  Correlation  Process 
Letters  to  other  consultants  (4) 
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MEI  ASSOCIATES,  INC. 

1050  WALTHAM  STREET.  LEXINGTON.  MASS.  '  2173 
(617)  862  3390 


August  11,  1989 


Dr.  Martha  C.  Poison 
Institute  of  Cognitive  Science 
University  of  Colorado 
Boulder,  Colorado  80309-0345 

RE:  Project  AIDA 

Dear  Martha: 

Now  that  the  three  kick-off  meetings  have  been  completed, 
we  would  like  to  summarize  your  task  assignment  for  cycle  1. 

As  you  proposed,  your  primary  task  is  a  combination  of  Tasks  1 
and  2 : 

1.  "State  the  ten  or  twelve  major  propositions  derived 
from  cognitive  learning  research  which  would  probably  be  agreed 
to  by  leading  investigators  in  the  field,  and  which  have  an 
essential  relation  to  instructional  design.  Describe  in  terms  as 
simple  as  possible  why  each  principle  is  important  to 
instructional  design,  and  what  effect  knowing  it  may  be  expected 
to  have  on  the  performance  of  the  instructional  designer." 

2.  "State  the  major  distinguishing  characteristics  of  the 
contents  of  cognition,  including  the  different  kind3  of  knowledge 
and  the  different  forms  each  type  may  take.  Consider  in  your 
description  the  contributions  of  epistemologists  (e.g.,  Michael 
Polanyi)  ana  cognitive  psychologists  (e.g.,  Anderson,  Simon). 
Restrict  your  answer  to  the  forms  of  cognition  which  carry  clear 
implications  for  the  nature  of  instruction  and  its  design.  As  an 
aim,  describe  what  kind  of  "knowledge  theory"  must  form  a  basis 
for  the  design  of  instruction." 

Tasks  1  and  2  will  be  correlated  with  lists  of  principles 
derived  from  Tasks  4,  5,  and  6,  as  3hown  in  the  attached 
schematic.  The  correlation  process  will  begin  when  you  submit 
your  draft  on  October  10  and  continue  until  we  send  the  integrated 
results  to  Harry  O'Neil  and  Charles  Reigeluth  on  October  31. 

Meanwhile,  as  recommended  by  the  consultants,  HRL  will 
concurrently  prepare: 


1 .  A  needs  statement 

2.  A  requirements  document 

3.  A  general  specification,  or  vision  of  AIDA 

On  another  subject,  Scott  Newcomb  says  the  JAG  at  Brooks  AFB 
has  declared  that  your  work  on  this  task  will  not  preclude  your 
bidding  on  any  future  procurements  connected  with  AIDA. 

Finally,  if  the  other  consultants  are  agreeable,  the  meeting 
in  October  will  be  moved  up  one  day  to  Tuesday  and  Wednesday, 
October  17,  18,  so  you  can  participate  in  the  HFS  meeting  on  the 
19th. 


We  believe  the  kick-off  meetings  have  been  productive  and 
augur  well  for  the  AIDA  project.  I  look  forward  to  receiving 
your  draft  on  October  10.  If  you  have  any  questions,  please  call 
me  directly  at  617-862-3390,  or  fax  617-862-5053. 


Sincerely  yours. 


Albert  E.  Hickey,  PhD 


Enel .  : 

Specifications  of  Consultant  Tasks 
Schematic  of  Correlation  Process 
Letters  to  other  consultants  (4) 
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ME  I  ASSOCIATES,  INC. 

1050  WALTHAM  STREET.  LEXINGTON,  MASS.  02173 
(617)  862-3390 


August  10,  1989 


Dr.  Robert  M.  Gagne 

233  East  Hickory  Knoll  Rd. 

Franklin,  North  Carolina  28734 

RE:  Project  AIDA 

Dear  Bob: 

Now  that  the  three  kick-off  meetings  have  been  completed, 
we  would  like  to  summarize  your  task  assignment  for  cycle  1. 

As  you  proposed,  your  task  is  Task  3:  "State  the  twenty  (or  so) 
principles  of  instructional  theory  (as  exhibited  for  example  in 
Reigeluth's  book)  which  can  be  commonly  held  by  all  instructional 
theorists,  and  concerning  which  there  is  minimal  or  zero 
disagreement,  except  perhaps  in  priority  of  importance.  Indicate 
common  ideas  and  variant  terminology;  identify  reasons  for 
differences  in  terminology  when  possible.  Consider  the  relative 
importance  of  these  principles  in  determining  the  nature  of 
instruction  and  the  process  of  instructional  design.  When 
appropriate,  indicate  differences  in  the  depth  of  knowledge  about 
learning  required  by  the  instructional  designer  when  using  each  of 
these  principles  in  designing  instruction." 

The  product  of  Task  3  will  be  correlated  with  the  results  of 
Tasks  4,  5,  and  6,  as  shown  in  the  attached  schematic.  The 
correlation  process  will  begin  when  you  submit  your  draft  on 
October  10  and  continue  until  we  send  the  integrated  results  to 
Harry  O'Neil  and  Charles  Reigeluth  on  October  31. 

Meanwhile,  as  recommended  by  the  consultants,  HRL  will 
concurrently  prepare: 

1.  A  needs  statement 

2.  A  requirements  document 

3.  A  general  specification,  or  vision  of  AIDA 

On  another  subject,  Scott  Newcomb  says  the  JAG  at  Brooks  AFB 
has  declared  that  your  work  on  this  task  will  not  preclude  your 
bidding  on  any  future  procurements  connected  with  AIDA. 


Finally,  if  is  agreeable  with  you,  the  meeting  in  October 


will  be  moved  up  one  day  to  Tuesday  and  Wednesday,  October  17, 

18,  to  avoid  a  conflict.  Martha  Poison  is  scheduled  to  chair  a 
panel  at  the  HFS  on  the  19th. 

We  believe  the  kick-off  meetings  have  been  productive  and 
augur  well  for  the  AIDA  project.  I  look  forward  to  receiving 
your  draft  on  October  10.  If  you  have  any  questions,  please  call 
me  directly  at  617-862-3390,  or  fax  617-862-5053. 


Sincerely  yours. 


Albert  E.  Hickey,  PhD 


Enel.  : 

Specifications  of  Consultant  Tasks 
Schematic  of  Correlation  Process 
Letters  to  other  consultants  (4) 
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ME  I  ASSOCIATES,  INC. 

1050  WALTHAM  STREET,  LEXINGTON.  MASS  0217] 
(617] 562-3390 


August  10,  1989 


Dr.  Henry  M.  Halff 
Halff  Resources  Inc. 

4913  33rd  Road,  North 
Arlington,  Virginia  22207 

RE:  Project  AIDA 

Dear  Henry- 

Now  that  the  three  kick-off  meetings  have  been  completed, 
we  would  like  to  summarize  your  task  assignment  for  cycle  1. 

As  you  proposed,  your  primary  task  is  to  "provide  a  concept  of 
how  AIDA  would  support  the  design  of  instruction  in  two  broad 
instructional  paradigms  to  be  selected.  And  identify  the 
principles  of  learning  and  instruction  that  apply  to  the  design 
of  the  two  AID As  and  to  their  instructional  products."  This  i3 
Task  4  on  the  attached  list. 

To  help  us  correlate  the  results  of  Tasks  4,  5,  and  6, 
please  make  a  separate  list  of  principles  derived  from  research 
on  cognition  and  cognitive  processes  involved  in  learning  and 
relevant  to  your  primary  task.  (This  is  akin  to  the  subtask 
described  in  the  second  sentence  of  your  own  task  definition.  It 
is  also  an  abbreviated  version  of  Gagne's  Tasks  1  and  2.) 

The  overall  process  of  correlation,  shown  in  the  attached 
schematic,  will  begin  when  you  submit  your  draft  on  October  10 
and  continue  until  we  send  the  integrated  results  to  Harry  O'Neil 
and  Charles  Reigeluth  on  October  31. 

Meanwhile,  as  recommended  by  the  consultants,  HRL  will 
concurrently  prepare: 

1.  A  needs  statement 

2 .  A  requirements  document 

3.  A  general  specification,  or  vision  of  AIDA 

On  another  subject,  Scott  Newcomb  says  the  JAG  at  Brooks  AFB 
has  declared  that  your  work  on  this  task  will  not  preclude  your 
bidding  on  any  future  procurements  connected  with  AIDA. 


Finally,  if  is  agreeable  with  you,  the  meeting  in  October 
will  be  moved  up  one  day  to  Tuesday  and  Wednesday,  October  17, 

18,  to  avoid  a  conflict.  Martha  Poison  is  scheduled  to  chair  a 
panel  at  the  HFS  on  the  19th. 

We  believe  the  kick-off  meetings  have  been  productive  and 
augur  well  for  the  AIDA  project.  I  look  forward  to  receiving 
your  draft  on  October  10.  If  you  have  any  questions,  please  call 
me  directly  at  617-862-3390,  or  faje  617-862-5053. 


Sincerely  yours. 


Albert  E.  Hickey,  PhD 


Enel.  : 
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ME  I  ASSOCIATES.  INC. 


KMO  WALTHAM  STREET  L£XINGTON.  MA  0217)  (617)  86M390 


December  IS,  1989 


RE:  Pro-ject  AIDA,  Your  task  in  cycle  2 


Following  the  meeting  at  HRL  last  week,  this  letter 
summarizes  your  task  assignment  for  cycle  2. 

As  you  recall,  the  goals  tor  cycle  2  are  to  (1)  define  the 
AIDA  concept,  (2)  define  the  functional  characteristics  of  AIDA, 
and  (3)  list  related  research  issues.  During  the  meeting,  a 
functional  block  diagram  was  used  to  organize  cycle  2  task3.  The 
core  functions  are  listed  below. 


1 .  Information 

1 . 1  Audience 

1 . 2  Environment 

1.3  Task 

2 .  Content 

3 .  Executive 

4 .  Strategies 

5 .  Delivery 

6 .  Evaluation 


In  the  enclosed  guidelines,  one  page  is  devoted  to  each  of  the 
six  functions.  The  questions  on  each  page  are  intended  to  bring  the 
AIDA  concept  to  a  new  level  of  detail  by  defining  the  variables/ 
attributes/characteristics  of  each  function  and  specifying  a  scale 
of  values  for  each  variable.  Please  answer  the  questions,  adding 
any  functions  or  variables  you  think  we  have  overlooked.  Answer  as 
many  of  the  questions  in  as  much  detail  as  time  allows. 

As  you  answer  the  questions,  please  point  out  those  questions 
you  think  raise  research  issues;  that  is,  questions  that  cauanot 
be  answered  completely  until  further  research  is  performed.  Make 
a  separate  list  of  specific  research  issues  in  need  of  attention. 

Also  enclosed  is  a  transcript  of  that  part  of  the  meeting  in 
which  the  cycle  2  tasks  were  discussed.  As  in  cycle  1,  these  are 
guidelines  only.  Do  not  let  them  limit  your  approach  to  defining 
the  functional  characteristics  of  AIDA  or  identifying  required 


research . 


You  have  been  allocated  eleven  days  to  perform  this  task,  plus 
two  days  to  attend  the  next  AIDA  Concept  Development  Meeting,  to  be 
held  at  AFHRL  February  20-21.  Please  send  me  your  cycle  2  results 
to  reach  here  by  Thursday,  February  15,  so  I  can  send  them  by  Federal 
Express  to  the  other  consultants  prior  to  the  meeting.  Call  me  if 
you  have  any  questions. 

See  you  in  sunny  Sam  Antonio  in  February.  As  before,  we 
have  made  reservations  for  you  at  the  Sheraton  Gunter  Hotel, 
guaranteed  for  late  arrival  on  February  19.  Meanwhile,  best  wishes 
for  the  holidays. 


Sincerely  yours. 


Albert  E.  Hickey,  PhD 
AIDA  Project  Manager 


An  AIDA  Concept 


Guidelines  for  the  cycle  2  task: 

Each  consultant  should  consider  the  attached  "AIDA  CONCEPT” 
diagram  and  the  questions  asociated  with  each  function 
identified. 


Each  consultant  should  respond  to  the  questions  related  to  some 
or  all  of  the  functions.  This  detailed  information  concerning  an 
AIDA  concept  should  facilitate  the  Phase  II  software  specification. 
Each  consultant  should  strive  for  internal  consistency  and  be 
less  concerned  with  whether  the  system  they  define  corresponds 
with  that  proposed  by  any  other  consultant.  During  the 
discussion  of  cycle  2  on  February  20-21  similarities  and 
differences  can  be  discussed  and  explored.  The  person  (s)  tasked 
with  the  specification  then  have  a  number  of  alternative  ideas 
from  which  to  choose. 

Two  places  probably  require  some  form  cf  expert  system  (Merrill' s 
opinion)  :  the  content  (task)  analysis  (knowledge  acquisition) 
function  and  the  executive  function.  The  form  of  the  rules  for 
the  executive  function  has  been  suggested,  but  the  form  of  the 
rules  for  the  content  (task)  analysis  functior  has  not.  The 
reason  is  that  content  analysis  could  take  a  number  of  forms  and 
the  nature  of  the  rules  (expertise)  involved  depends  on  which 
approach  is  used. 

For  every  function  only  end  information  is  requested  by  the 
question.  For  example,  during  the  information  gathering  on 
students  an  important  characteristic  may  be  "student  motivation". 
The  values  could  be  scaled  on  a  10  point  scale  with  10  being 
very  motivated  and  a  1  being  negatively  motivated.  This  value 
for  motivation  could  be  used  in  the  executive  function  to 
determine  some  strategy  or  some  parameter  value  for  some 
strategy.  However,  the  user  (designer  or  SME)  may  not  know  how 
to  determine  learner  motivation.  At  this  point  a  series  of 
attributes  or  conditions  that  affect  the  learner,  e.g.,  job 
change,  increase  in  pay,  required  assignment  cnange,  etc.,  could 
be  evaluated  and  a  set  of  rules  could  lead  AIDA  to  suggest  the 
motivation  level  for  the  student. 

Any  piece  of  information  requested  by  the  system  could  have 
associated  with  it  a  "mini-expert"  to  help  select  the  appropriate 
value  for  this  information.  The  consultants  snould  indicate  what 
other  attributes  and  values,  known  or  hypothesized,  might  lead  to 
a  given  value  on  a  characteristic  used  in  the  executive. 


INFORMATION  CONTENT 


INFORMATION  CONTENT  (4)  STRATEGIES: 


ME  I  ASSOCIATES,  INC. 

1050  waltham  street.  Lexington,  mass.  02173 

(617)  862-3390 


March  8,  1990 


Dr.  Harold  F.  O'Neil 
Advance  Design  Information 
15366  Longbow  Drive 
Sherman  Oaks,  California  91403 

RE :  Project  AIDA,  Your  task  in  cycle  2 

Dear  Harry: 

Further  to  my  letter  dated  March  1,  enclosed  please  find  the 
following  documents: 

(1)  AIDA  Progress  Review  dtd  28  Feb  90 

(2)  AIDA  Long  Range  Research  and  Development  Plan  (Draft) 

(no  date) 

(3)  Task  Order  0006  Final  Report  (Draft )  dtd  2  Mar  90 


As  your  task  in  cycle  2,  you  are  requested  to: 

1.  Review  the  six  cycle  2  concept  papers  sent  to  you  on  March  1; 
the  AIDA  Progress  Review;  the  AIDA  Long  Range  R&D  Plan 
(Draft) ;  and  section  V.A,  V.B  and  VI. A  of  the  Task  Order 
0006  F inal  Report  (Draft) . 

2.  Provide  a  critique  of  the  items  cited  in  (1)  with  regard  to 
AIDA's  R&D  coherence,  soundness  of  investment  strategy,  and 
potential  for  success.  Of  special  interest  are  alternative 
but  promising  conceptions  of  AIDA. 

3.  Provide  an  elaboration  of  sections  VI. A  and  VI. B  of  the 
Task  Order  0006  Final  Report  (Draft)  based  on  your 
review  and  critique. 

4.  Identify  pertinent  research  issues  for  sections  VII. A  and 
VII. B  of  the  Task  Order  0006  Final  Report  (Draft) . 

We  have  allocated  5  days  for  you  to  do  this  work  and  look 
forward  to  receiving  your  results  on  or  before  April  18  so  that 
we  can  distribute  your  paper  to  the  other  consultants  prior  to 
the  next  meeting,  scheduled  for  April  24-25  in  San  Antonio. 

(The  kick-off  meeting  for  Phase  II  (Task  0013)  is  scheduled 
for  April  26 . ) 
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Aside  from  preparation  of  the  final  report,  this  is  the  last 
activity  in  Task  0006.  Your  paper  will  have  an  important  effect 
on  the  future  of  AIDA. 

Please  call  me  directly  if  you  have  any  questions.  Since 
I  will  be  out  of  the  country  until  March  26,  however,  please  feel 
free  to  call  Dan  Muraida  or  Mike  Spector  (512-536-2242)  in  my 
absence . 

I  look  forward  to  seeing  you  in  San  Antonio  in  April. 
Meanwhile,  you'll  receive  a  second  letter  soliciting  your 
participation  as  a  consultant  in  AIDA  Phase  II  (Task  0013)  . 


Sincerely  yours. 


Albert  E.  Hickey,  PhD 
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MEI  ASSOCIATES.  INC. 
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APPENDIX  C 


LIST  OF  DOCUMENTS  PRODUCED  IN  TASK  0006 


Description 


Date 


Pages 


Administration 


Request  for  Proposal 


24  Feb  89 


Preliminary  Design  Considerations  for  an  AIDA 
J.M.  Spector 

Refinement  Considerations  for  an  Advanced 
Instructional  Design  Advisor 

J.M.  Spector  22  Aug  89 

AF  Task  Description/Specification,  Task  Order  0006., 
Title:  Specif ications  for  an  Advanced  Instructional 
Design  Advisor _ (AIDA)  for  CBT 


Task  Order  Proposal,  Mei  Associates,  Inc. 


31  Mar  89 


Technical  Task  Description  (Revised  11  July  89J_ 

11  Jul  89 


Agenda 


Agenda:  Kick-off  Meeting,  Part  I 
Agenda:  Kick-off  Meeting,  Part  II 
Agenda:  Kick-off  Meeting,  Part  III 
Agenda:  First  Concept  Design  Meeting 
Agenda:  Second  Concept  Design  Meeting 
Agenda:  Third  Concept  Design  Meeting 
Agenda:  Final  Concept  Review  Meeting 


6  Jul  89 
18-19  Jul  89 
8-  9  Aug  89 
17-18  Oct  89 
6-  7  Dec  89 
20-21  Feb  90 
24-25  Apr  90 


Transcripts  of  Selected  AIDA  Meetings 

Kick-off  Meeting,  Part  II 
Kick-off  Meeting,  Part  III 
First  Concept  Design  Meeting 

AIDA  Needs  Assessment  Report 

D.  Muraida  &  J.M.  Spector 

Task  Assignments 


18-19  Jul  89 
8-  9  Aug  89 
17-18  Oct  89 


2-6  Oct  89 


AIDA  Cycle  1  Task  Assignments:  Design  Consultants  (5) 
AIDA  Cycle  1  Task  Assignments:  Review  Consultants  (2) 
AIDA  Cycle  2  Task  Assignments:  Design  Consultants 


2  ea. 


AIDA  Cycle  2  Task  Assignments:  Review  Consultants 


AIDA  Concent  Papers.  Cycle  1 

Gagne,  R.:i.  ari:.rio]  3^  cf  ~rr.strr  cticr.al  Theory 

1  Oct  89  5 

Kintsch,  E.  Principles  of  Instructional  Theory  from 

Research  on  Human  Cognition  11 

Poison,  M.C.  Cognitive  Theory  as  a  Basis  for  Instruction 

Oct  89  20 

Halff,  H.M.  Prospects  for  Automating  Instructional  Design 

Oct  89  26 

Merrill,  M. ,  Project  AIDA:  A  Concept  Paper 
Li ,  Z .  ,  and 

Jones,  M.  Sep  89 

Merrill,  M.D.  Project  AIDA:  A  Theory  Paper 


26 


Sep  89 


12 


Tennyson,  R.D.  Cognitive  Model  of  Learning  and  Cognition 

11  Oct  89  5 

Tennyson ,  R.D.  Cognitive  Science  Update  of  Instructional  Systems 
Desicn  Models  Oct  89  66 

AIDA  Cycle  1  Critigues/Reviews 

O'Neil,  H.F.  Cycle  1  Critique 

Reigeluth,  C.M.AIDA  Svnthesis/Critigue 


6-7  Dec  90 


45 


30  Nov  90 


Muraida,  D.  Review  of  Cycle  1  Papers 
&  Spector,  J.M.  5  Dec  89 

AIDA  Concept  Papers.  Cycle  2 

Gagne,  R.M.  AIDA  -  Concept  of  Operation 


22 


Friedman,  A.  Designing  Graphics  for  Courseware:  The  Role  of 
Graphics  in  Knowledge  and  Skill  Acouisition 

Dec  89  54 


Halff,  H.M.  Automating  Instructional  Design 

and  Development  Feb  90 


36 


NO. 

1 

10 

Aug 

89 

8 

No. 

2 

27 

Oct 

89 

12 

No. 

3 

17 

Dec 

89 

13 

No. 

4 

28 

Feb 

90 

21 

No. 

5 

20 

Apr 

90 

19 
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APPENDIX  D.  O'NEIL'S  AIDA  FEATURES  EVALUATION 


PRELIMINARY  RESULTS 

The  forms  were  completed  during  the  April  AIDA  Review  Meeting. 
Respondents  used  LOW,  MEDIUM,  and  HIGH  to  indicate  the  relative 
merit  of  22  potential  AIDA  features  with  respect  to  11  areas. 
Respondents  were  grouped  with  regard  to  their  role  in  the 
training  process:  Managers,  Instructional  Designers,  and 
Researchers.  Results  were  averaged  using  a  value  of  2  for  LOW,  6 
for  MEDIUM,  and  10  for  HIGH.  Averages  are  presented  below.  It 
should  be  noted  that  in  general  there  were  not  enough  respondents 
in  category  11  to  place  any  confidence  those  results. 

The  22  features  are  lettered  A  through  V  and  represent  the 
following: 

A  -  templates 

B  -  automated  ISD  paperwork 

C  -  cognitive  science  augmentation 

D  -  instructor  aids 

E  -  computerized  measurement 

F  -  type  of  media  (multi-media) 

G  -  formative  evaluation  tools 

H  -  updated  ISD  model 

I  -  updated  learning  theory 

J  -  author  management  system 

K  -  cost-effectiveness  tool 

L  -  task  database  tool 

M  -  catalog  of  courseware 

N  -  catalog  of  author  aids 

O  -  research  justification  (explanation  feature) 

P  -  ITS  for  ISD 

Q  -  intelligent  job  aid 

R  -  intelligent  template  interfaces 

S  -  intelligent  help 

T  -  hyper-media 

U  -  on-line  documentation 

V  -  portability 

The  11  areas  are  numbered  1  through  11  as  follows: 

1  -  technical  merit 

2  -  value  to  the  SME 

3  -  value  to  the  instructional  designer 

4  -  state-of-the-art  technology 

5  -  R&D  costs 

6  -  implementation  costs 

7  -  risk 

8  -  Air  Force  gain/ impact 

9  -  contribution  to  science  and  technology 

10  -  joint  service  utility 

11  -  feasibility  to  complete  as  resourced 


O'NEIL'S  EVALUATION  RESULTS  (8  RESEARCHERS) 
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